Notes by Alexey Diyan
☰

Summary of passed 2014 year

2014 year had passed and it would be nice to dump all things that I did that year.

Major

Very first year with 0 days usage of Windows

No usage at home, no usage at work. For a long time I was very interested in Linux and Open Source. During my work in .NET ecosystem I have a strong feeling that Open Source libraries and tools fit best to my way of getting job done.

In September 2011 I made a major shift from .NET/Windows to Python/Linux. I did spent 6 years on .NET platform and have not slightest doubts to do this big change.

From Ubuntu to Arch Linux

For about one and a half year I’ve put a large amount of effort to learn main Ubuntu administration topics - coreutils, boot loaders, file systems, daemons, networking, logging, security, etc.

In parallel, a lot of desktop environment were tried and changed - Unity, Gnome 2, KDE 4, XFCE, LXDE and eventually I ended up on a very lightweight tiling window manager - i3wm.

Those two things and also a willing to have bleeding edge software led me to Arch Linux which I have started to use at home starting from October 2014.

Medium/Minor

Other things are somewhat smaller, so I will just list them without any ordering.

Visited Crete and Thailand. I’m rather indifferent to traveling, but my wife is crazy about this. To make her happier I had to visit United Arab Emirates, Sri Lanka, Turkey, Poland, Czech Republic, Germany, Austria so far. :)

Got B1/B2 USA Visa which will be valid for next 10 years.

Docker user since v0.4. Using it for software evaluation, integration testing, setting up dev environment (including boot2docker on Mac OS for one my colleague). Planning to start use it in production this year.

Become an advocate for structured logging - structlog, nxlog, logstash, Kibana (Ruby version, then AngularJS). Now considering to change nxlog + Logstash into Mozilla’s Heka.

Become a metrics junky - our Graphite installation contains metrics from application (scales then metrology), Nagios (graphios), CollectD, Amazon CloudWatch (homegrown script). Now planning to switch from Graphite to InfluxDB and Grafana.

Did learn how to write LSB compatible init.d scripts and then happily threw away all that knowledge and learn Upstart. I would rather support two scripts for Upstart and SystemD instead of returning to the Middle Ages. Learning SystemD.

Started track all configs for nginx, upstart, logrotate, gunicorn, uwsgi, heka, git deploy/develop hooks in Git repository next to code. They are deployed on each web node via post-update/post-merge git hooks. Hopefully I will start use Ansible for that at some point.

Learn Nginx a bit - compression, etags, satic file caching with and w/o expire validation, PAM/LDAP integration, ACLs.

Invested some time to learn Golang, keeping most attention on a philosophy benind this language and ecosystem in general. Right now it’s mostly theoretical knowledge and hopefully this year I would start some project in Golang.

My chronic test-all-the-things infection became worse this year, so I’ve adopted couple practices:

  • assert_json_equal function that displays unified diff along with AssertionError
  • In memory Gevent-SocketIO server inspired on this implementation
  • Asked my teammate to develop in process RiakClientMock that supports get/set and 2i lookups (unfortunately, later we dropped that code due to switch on Cassandra)
  • Came up with _cassandra_process (build/start Docker container with real Cassandra node once per test session) and cassandra_mock (truncate all tables before each test function/method) py.test fixtures
  • Started use in process mockredispy instead of real redis instance to speed up tests (unfortunately it does not emulate pub/sub but we are planning to change that)
  • Evaluated approach with Dockerized Selenium Server with PhantomJS, SlimerJS, Firefox and Chrome for running frontend tests (planning to integrate that into dev our process in 2015)

Start using a bunch of nice Python libraries/tools:

Did a release of pywinrm v0.0.3. My pesonal project is moving forward - I was happy to see a code contribution from about 7 developers. pywinrm has about 2000-2500 monthly downloads on PyPI and it was included into Ansible project.

Got acquainted with several Virtual DOM and/or FRP frameworks - VueJS, ReactJS, Mithrill, Mercury, Virtual-DOM. I hate JavaScript but planning to write some code in ECMAScript 6 with one of those frameworks (most likely Mercury). Even if I’m pure backend developer I’m prefer keeping fresh skills in JavaScript at least on a very basic level.

Used awesome Percona Tookit (pt-archiver, pt-config-diff, pt-mysql-summary, pt-online-schema-change, pt-table-sync) for managing pretty big MySQL servers that contains about 600GB data.

Dived into a world of distributed databases/storages - Riak, Cassandra, Couchbase, BigCouch, Hadoop, HBase, CRDT, CAP theorem, Paxos, Raft and all that staff. Most time I have spent with Riak, Cassandra but ended up with Cassandra for my current project. Also I did tried Presto/Cassandra and planning to work with Elasticsearch. Spark/Shark/Cassandra, Spark SQL/Cassandra this year.

Also I was involved into setting up PCI DSS compliant environment. During that time I have learn a lot best practices on security/traceability/configration management/monitoring that could be applied to any project. Amout of resources that I’ve collected was pretty big, so I published everything on Gist.

2014 was awesome year with a lot of interesting tasks and technical chalenges and I’m eager to try new things that will come in this 2015 year.

Happy New Year!

(better later than never :)

Written by
Later article
On Virtual DOM, FRP, Hyperscript