I recently gave a talk on SPDY, the new protocol which will serve as the foundation for HTTP 2.0. SPDY introduces some interesting features to solve current limitations with how HTTP 1.1 sits on top of TCP. Check out the deck for a high-level overview, with links..
When making a choice in the tech world there are two wide-spread approaches: “What’s better, X or Y?” and “Should I use xyz?”. The “or” debate is always an entertaining topic usually ending in an absurdly hilarious flame war. The “Should I use xyz?” is a subtler, more prevalent question in the tech community leading to an extensive amount of discourse. Fairly rational, usually with some good insight, but still a time consuming task. I’ve fallen victim to both approaches when exploring a technology decision. What I realized is I’m asking the wrong question. There are only two things I should ask:
1) What problem do I need to a solve?
2) How do I want to solve it?
Once I take this approach I have an opinionated basis for decision-making and I have a clear direction in how to make that decision. Frameworks–web or javascript–are excellent examples on taking this approach. Most of these frameworks were born on the simple premise of solving a problem in an opinionated way. Backbone takes a bare-bones approach to a front-end, event-driven structure; Ember offers a robust, “things just happen” framework. Sinatra and co. offers an http-first approach to development. Rails and variants are opinionated in web application structure. Do you agree with that approach? Yes, excellent! No? Find something else or roll your own.
Don’t know the answer? That’s okay too. Most beginners want to make the “right” choice on what to learn. But the thing is there is no “right” answer. For a beginner choosing python vs. ruby vs. php vs scala wastes effort. Just build something using something: you’ll soon develop your own opinions, with “how easy is this to learn” probably the first. Next, when your rails codebase is out of control and you’re drowning in method_missing issues maybe you’ll want a more granular, service-orientated approach and the type-safety of Scala. Maybe not… But you’ll have a valid problem to solve and a reasonable opinion to go with it.
I suggest reading Andrew Alexeev’s reason on why he built NGINX and Poul-Henning Kamp’s rationale on how you write a modern application. Like so many others these incredible open-source systems were born from a problem and the way someone wanted it solved. But those systems didn’t happen overnight and the authors didn’t start from scratch. They spent years encountering, learning, and dealing with problems in their respective spaces. They knew the problem domain well, they knew how they wanted the problem solved, and they solved it.
So put your choice in a context and don’t sweat the details which are irrelevant to the task at hand. When you need to know those details you’ll know them, and when you hit problems you’ll know how you want them solved.
As much as I wish a LinkedIn profile could be a substitute for a resume, it’s not, and I needed an updated resume. My previous resume was done some time ago with InDesign when I was on a design-tools kick. It worked well, but InDesign isn’t the best choice for a straight forward approach to a resume and I was not interested in going back to word. So in honor of my friend Karthik’s programming themed resume I had an idea: program my resume. My requirements were simple:
I’m a big fan of Markdown and happy to see the prevalence of Markdown across the web, however fragmented. I use Markdown to publish this blog and felt it would work well for writing a resume. The only problem is layout: you have minimal control over structural html elements which can make aspects of design difficult. For writing articles this isn’t a problem but when you need structural markup for CSS it can be limiting. Luckily I found Maruku, a ruby-based markdown interpreter which supports PHP Markdown Extra and a new meta-data syntax for adding id, css, and div elements to a page. It does take away from Markdown’s simplicity but adds enough structure for design. Combined with CSS I had everything I needed to fulfill my requirements.
My markdown resume is on GitHub. I was surprised it rendered well with GitHub-Flavored Markdown despite the extraneous Maruku elements. I knew I was on the right track. Maruku lets you add your own stylesheets to the html output which I used for posting online. One simple command gets me from markdown to ready-to-publish html. Exactly what I wanted.
Markulu supports pdf output as well, but requires a heavy LaTex install which I wasn’t happy with. I also wasn’t impressed with the LaTex PDF output. Luckily there’s an easy alternative: printing to PDF. I used some SASS media query overrides on top of Html 5 Boilerplate’s default styles to control the print layout in the way I wanted. You can even specify page breaks and print margins via CSS. I favored Safari’s pdf output over Chrome’s for the sole reason Safari automatically embedded custom fonts in the final PDF.
At the end of the day I realized I probably didn’t need to add explicit divs to Markdown; I could have gotten the layout I wanted with just vanilla Markdown and CSS3 queries. I also could have a semantically better markup if I used HAML to add
At the end of the day, it’s all a tradeoff. GitHub flavored markdown, Markdown Here and other interpreters support fenced code blocks; I like the idea of adding fenced blocks to get
For the past few years this blog ran apache + mod_php on an ec2-micro instance. It was time for a change; I’ve enjoyed using nginx in other projects and thought I could get more out of my micro server. I went with a php-fpm/nginx combo and am very surprised with the results. The performance charts are below; for php the response times varied little under minimal load, but nginx handled heavy load far better than apache. Overall throughput with nginx was phenomenal from this tiny server. The result for static content was even more impressive: apache effectively died after ~2000 concurrent connections and 35k total pages killing the server; nginx handled the load to 10,000 very well and delivered 160k successful responses.
Here’s the loader.io results from static content from http://www.michaelhamrah.com, comparing apache with nginx. I suggest clicking through and exploring the charts:
Apache only handled 33.5k successful responses up to about 1,300 concurrent connections, and died pretty quickly. Nginx did far better:
160k successful response with a 22% error rate and avg. response time of 142ms. Not too shabby. The apache run effectively killed the server and required a full reboot as ssh was unresponsive. Nginx barely hiccuped.
The results of my wordpress/php performance is also interesting. I only did 1000 concurrent users hitting blog.michaelhamrah.com. Here’s the apache result:
There was a 21% error rate with 13.7k request served and a 237ms average response time (I believe the lower average is due to errors). Overall not too bad for an ec2-micro instance, but the error rate was quite high and nginx again did far better:
A total of 19k successes with a 0% error rate. The average response time was a little higher than apache, but nginx did serve far more responses. I also get a kick out of the response time line between the two charts. Apache is fairly choppy as it scales up, while nginx increases smoothly and evens out when the concurrent connections plateaus. That’s what scalability should look like!
There are plenty of guides online showing how to get set up with nginx/php-fpm. The Nginx guide on WordPress Codex is the most thorough, but there’s a straightforward nginx/php guide on Tod Sul. I also relied on an nginx tuning guide from Dakini and this nginx/wordpress tuning guide from perfplanet. They both have excellent information. I also think you should check out the html5 boilerplate nginx conf files which have great bits of information.
If you’re setting this up yourself, start simple and work your way up. The guides above have varying degrees of information and various configuration options which may conflict with each other. Here’s some tips:
Decide if you’re going with a socket or tcp/ip connection between nginx + php-fcm. A socket connection is slightly faster and local to the system, but a tcp/ip is (marginally) easier to set up and good if you are spanning multiple nodes (you could create a php app farm to compliment an nginx front-facing web farm).
I chose to go with the socket approach between nginx/php-fpm. It was relatively painless, but I did hit a snag passing nginx requests to php. I kept getting a “no input file specified” error. It turns out it was a simple permissions issue: the default php-fpm user was different the nginx user the webserver runs under. Which leads me to:
Plan your users. Security issues are annoying, so make sure file and app permissions are all in sync.
Check your settings! Read through default configuration options so you know what’s going on. For instance you may end up running more worker processes in your nginx instance than available cpu’s killing performance. Well documented configuration files are essential to tuning.
Plan for access and error logging. If things go wrong during the setup, you’ll want to know what’s going on and if your server is getting requests. You can turn access logs of later.
Get your app running, test, and tune. If you do too many configuration settings at once you’ll most likely hit a snag. I only did a moderate amount of tuning; nginx configuration files vary considerably, so again it’s a good idea to read through the options and make your own call. Ditto for php-fcm.
I am really happy with the idea of running php as a separate process. Running php as a daemon has many benefits: you have a dedicate process you can monitor and recycle for php without effecting your web server. Pooling apps allows you to tune them individually. You’re also not tying yourself to a particular web server; php-fpm can run fine with apache. In TCP mode you can even offload your web server to separate node. At the very least, you can distinguish php usage against web server usage.
So my only question is why would anyone still use apache?
I was shocked to learn the number of sites which failed to handle the spike in web traffic during the Super Bowl. Most of these sites served static content and should have scaled easily with the use of CDNs. Scaling sites, even dynamic ones, are achievable with well known tools and techniques.
At a basic level accessing a web page is when one computer, the client, connects to a server and downloads some content. A problem occurs when the number of people requesting content exceeds the ability to deliver content. It’s just like a restaurant. When there are too many customers people must wait to be served. Staff becomes stressed and strained. Computers are the same. Excessive load causes things to break down.
To handle more requests there are three things you can do: produce (render) content faster, deliver (download) content faster and add more servers to handle more connections. Each of these solutions has a limit. Designing for these limits is architecting for scale.
A page is composed of different types of content: html, css and js. This content is either dynamic (changes frequently) or static (changes infrequently). Static content is easier to scale because you create it once and deliver it repeatedly. The work of rendering is eliminated. Static content can be pushed out to CDNs or cached locally to avoid redownloading. Requests to origin servers are reduced or eliminated. You can also download content faster with small payload sizes. There is less to deliver if there is less markup and the content is compressed. Less to deliver means faster download.
Dynamic content is trickier to cache because it is always changing. Reuse is difficult because pages must be regenerated for specific users at specific times. Scaling dynamic content involves database tuning, server side caching, and code optimization. If you can render a page quickly you can deliver more pages because the server can move on to new requests. Most often, at scale, you want to treat treat dynamic content like static content as best you can.
Adding more servers is usually the easiest way to scale but breaks down quickly. The more servers you have the more you need to keep in sync and manage. You may be able to add more web servers, but those web servers must connect to database servers. Even powerful database servers can only handle so many connections and adding multiple database servers is complicated. You may be able to add specific types of servers, like cache servers, to achieve the results you need without increasing your entire topology.
The more servers you have the harder it is to keep content fresh. You may feel increasing your servers will increase your load. It will become expensive to both manage and run. You may be able to achieve a similar result if you cut your response times which also gives the end user a better experience. If you understand the knobs and dials of your system you can tune properly.
Don’t be afraid to make assumptions about your traffic patterns. This will help you optimize for your particular situation. For most publicly facing websites traffic is anonymous. This is particularly true during spikes like the Super Bowl. Because you can deliver the same page to every anonymous user you effectively have static content for those users. Cache controls determine how long content is valid and powers HTTP accelerators and CDNs for distribution. You don’t need to optimize for everyone; split your user base into groups and optimize for the majority. Even laxing cache rules on pages to a minute can shift the burden away from your application servers freeing valuable resources. Anonymous users will get the benefit of cached content with a quick download, dynamic users will have fast servers.
You can also create specific rendering pipelines for anonymous and known users for highly dynamic content. If you can identify anonymous users early you may be able to avoid costly database queries, external API calls or page renders.
HTTP powers the web. The better you understand HTTP the better you can leverage tools for optimizing the web. Specifically look at http cache headers which allow you to use web accelerators like Varnish and CDNs. The vary header will allow you to split anonymous and known users giving you fine grained control on who gets what. Expiration headers determine content freshness. The worst thing you can do is set cache headers to private on static content preventing browsers from caching locally.
Varnish is an HTTP accelerator. It caches dynamic content produced from your website for efficient delivery. Web frameworks usually have their own features for caching content, but Varnish allows you to bypass your application stack completely for faster response times. You can deliver a pre-rendered dynamic page as if it were a static page sitting in memory for a greater number of connections.
Edge Side Includes allow you to mix static and dynamic content together. If a page is 90% similar for everyone, you can cache the 90% in Varnish and have your application server deliver the other 10%. This greatly reduces the work your app server needs to do. ESI’s are just emerging into web frameworks. It will play a more prominent role in Rails 4.
You don’t need to add more servers to your own data center. You can leverage the web to fan work out across the Internet. I talk more about CDN’s, the importance of edge locations and latency in my post Building for the Web: Understanding the Network.
Your application servers should be reserved for doing application-specific work which is unique to every request. There are more efficient ways of delivering the same content to multiple people than processing a request top-to-bottom via a web framework. Remember “the same” doesn’t mean the same indefinitely; it’s the same for whatever timeframe you specify.
If you run Varnish servers in multiple data centers you can effectively create your own CDN. Your database and content may be on the east coast but if you run a Varnish server on the west coast an anonymous user in San Fransisco will have the benefit of a fast response time and you’ve saved a connection to your app server. Even if Varnish has to deliver 10% dynamic content via an ESI on the east coast it can leverage the fast connection between data centers. This is much better then the end user hoping coast-to-coast themselves for an entire page.
Amazon’s Route 53 offers the ability to route requests to an optimal location. There are other geo-aware DNS solutions. If you have a multi-region setup you are not only building for resiliency your are horizontally scaling your requests across data centers. At massive scale even load balancers may become overloaded so round-robin via DNS becomes essential. DNS may be a bottleneck as well. If your DNS provider can’t handle the flood of requests trying to map your URL to your IP address nobody can even get to your data center!
If you can take an action when things get rough you can better handle spikes. Auto scaling groups are a great feature of AWS when some threshold is maxed. If you’re not on AWS good monitoring tools will help you take action when things hit a danger zone. If you design your application with auto-scaling in mind, leveraging load balancers for internal communication and avoiding state, you are in a better position to deal with traffic growth. Scaling on demand saves money as you don’t need to run all your servers all the time. Pinterest gave a talk explaining how it saves money by reducing its server farm at night when traffic is low.
Page sizes can be greatly reduced if you enable compression. Web traffic is mostly text which is easily compressible. A 100kb page is a lot faster to download than a 1mb page. Don’t forget about internal communication as well. In todays API driven world using efficient serialization protocols like protocol buffers can greatly reduce network traffic. Most RPC tools support some form of optimal serialization. SOAP was the rage in the early 2000s but XML is one of the worst ways to serialize data for speed. Compressed content allows you to store more in cache and reduces network I/O as well.
A performance bottleneck may be caused by one particular feature. When developing new features, especially on a high traffic site, the ability to shut down a misbehaving feature could be the quick solution to a bad problem. Most high-traffic websites “leak” new features by deploying them to only 10% of their users to monitor behavior. Once everything is okay they activate the feature everywhere. Similar to determining page freshness for caches, determining available features under load can keep a site alive. What’s more important: one specific feature or the entire system?
Asynchronous programming is a challenge and probably a last-resort for scaling. Sometimes servers break down without any visible threshold. You may have seen a slow request but memory, cpu, and network levels are all okay. This scenario is usually caused by blocking threads waiting on some form of I/O. Blocked threads are plugs that clog your application. They do nothing and prevent other things from happening. If you call external web services, run long database queries or perform disk I/O beware of synchronous operations. They are bottlenecks. Asynchronous based frameworks like node.js put asynchronous programming at the forefront of development making them attractive for handling numerous concurrent connections. Asynchronous programming also paves the way for queue-based architectures. If every request is routed through a queue and processed by a worker the queue will help even out spikes in traffic. The queue size will also determine how many workers you need. It may be trickier to code but it’s how things scale.
When dealing with a high-load environment nothing can be off the table. What works for a few thousand users will grow out of control for a few million. Even small issues will become exponentially problematic.
Scaling isn’t just about the tools to deal with load. It’s about the decisions you make on how your application behaves. The most important thing is determining page freshness for users. The decisions for an up-to-the-second experience for every user are a lot different than an up-to-the-minute experience for anonymous users. When dealing with millions of concurrent requests one will involve a lot of engineering complexity and the other can be solved quickly.
A few months ago I helped a developer looking to better embrace test driven development. The session was worthwhile and made me reflect on my journey with TDD.
Writing tests is one thing. Striving for full test coverage, writing tests first and leveraging integration and unit tests is another. Some people find writing tests cumbersome and slow. Others may ignore tests for difficult scenarios or code spikes. When first working with tests I felt the same way. Over time I worked through issues and my feeling towards TDD changed. The pain was gone and I worked more effectively.
TDD is about speed. Speed of development and speed of maintenance. Once you leverage TDD as a way to better produce code you’ve unlocked the promise of TDD: Code more, debug less.
How many times have you verified something works by firing up your browser in development? Too many times. You build, you wait for the app to start, you launch the browser, you click a link, you fill in forms, you hit submit. Maybe there’s a breakpoint you step through or some trace statements you output. How much time have you wasted going from coding to verifying your code works? Too much time.
Stay in your editor. It has everything you need to get stuff done. Avoid the context switch. Avoid repetitive typing. Have one window for your code and another for your tests. Even on small laptops you can split windows to have both open at once. Gary Bernhardt, in an excellent Peepcode, shows how he runs specs from within vim. Ryan Bates, in his screencast How I Test, only uses the browser for UI design. If you leave your editor you are wasting time and suffering a context switch.
Every language has some sort of continuous testing runtime. Detect a file change, run applicable tests. Take a look at Guard. Selenium and company are excellent browser testing tools. Jasmine works great for Javascript. Rspec and Capybara are a solid combination. Growl works well for notifications. By staying in your editor you are coding all that manual verification away. Once coded you can repeat indefinitely.
Test driven doesn’t mean test after. This may be the hardest rule for newcomers to follow. We’ve been so engrained to write code, to design classes, to focus on OOP. We know what we need to do. We just need to do it. Once code works we’ll then write tests to ensure it always works. I’ve done this bad practice myself.
When you test last you’re missing the why. Customer gets welcome email after signing up means nothing without context. If you know why this is needed you are in a better position to define your required tests and start shaping your code. The notification could be a simple acknowledgement or part of some intricate flow. If you know the why you are not driving blind. The what you will build and the how you will build it will follow. If you code the other way around, testing later, you’re molding the problem to your solution. Define the problem first, then solve succinctly.
One of my favorite newbie mistakes is when a developer writes some code, then writes a test, watches the test pass, then is surprised when the code fails in the browser. But the test passed!
Anyone can write a green test. It is the action of going from red to green which gives the test meaning. Something needs to work, it doesn’t. Red state. You change your code, you make it work. Green state. Without the red state first you have no idea how you got to a green state. Was it a bug in your test? Did you test the right thing? Did you forget to assert something? Who knows.
Combined with the why going from red to green gives the code shape. You don’t need to over-think class design. The code you write has purpose: it implements a need to make something work that doesn’t. As your functionality becomes more complex, your code becomes more nimble. You deal with dependencies, spawning new tests and classes when cohesion breaks down. You stay focused on your goal: make something work. Combined with git commits you have a powerful history to branch and backtrack if necessary. As always, don’t be afraid to refactor.
Testing first also acts as a safeguard. Too often developers will pull work from a backlog prematurely. They’ll make assumptions, code to those assumptions, and have to make too many changes before release. If the first thing you do after pulling a story is ask yourself “how can I verify this works” you’re thinking in terms of your end-user. You’re writing acceptance tests. You understand what you need to deliver. BDD tools like Cucumber put this paradigm in the foreground. You can achieve the same effect with vanilla integration tests.
Most of the time not testing comes down to two reasons. The code is too hard to test or the code is not worth testing. There are other reasons, but they are all poor excuses. If you want to test code you can test code.
Code shouldn’t be too hard to test. Testing distributed, asynchronous systems is hard but still testable. When code is too hard to test you have the wrong abstraction. You’re API isn’t working. You aren’t adhering to SOLID principles. Your testing toolkit isn’t sufficient.
Static languages can rely on dependency injection to handle mocking, dynamic languages can intercept methods. Tools like VCR and Cassette can fake http requests for external dependencies. Databases can be tested in isolation or faked. Asynchronous code can be tricky to test but becomes easier when separating pre and post conditions (you can also block in unit tests to handle synchronization).
The code you don’t test, especially difficult code, will always bite you. Taking the time to figure out how to test will clean up the code and will give you incredible insight into how your underlying framework works.
I worked with a developer that didn’t write tests because the requirements, and thus code, were changing too much and dealing with the failing tests was tedious. It actually signified a red flag exposing larger issues in the organization but the point is a common one. Some developers don’t test because code may be thrown out or it’s just a spike and not worth testing.
If you’re not testing first because it’s a faster way to develop, realize that there is no such thing as throw away code (on the other hand, all code is throw away code). Mixing good, tested code with untested code creates technical debt. If you put a drop of sewer in a barrel of wine you will have a barrel of sewer. The code has no why. It may be just a spike but it could also turn out to be the next best thing. Then you’re left retrofitting unit tests, fitting a square peg in a round hole.
Once you start testing first a lot of pieces fall into place. The balance between integration and unit tests is an interesting topic when dealing with code coverage. There will be overlap in code coverage but not in terms of covered functionality.
Unit tests are the distinct pieces of your code. Integration tests are how those pieces fit together. You have a customer class and a customer page. The unit tests are the rules around the customer model or the distinct actions around the customer controller. The integration tests are how the end user interacts with those models top to bottom. Pivotal Labs talks about changing state in cucumber steps showing how integration tests monitor the flow of events in an application. Unit tests are for the discrete methods and properties which drive those individual events.
Developing applications is much more than coding. Focusing on tools and techniques at your disposal will help you write code more effectively. Your IDE, command line skills, testing frameworks, libraries and development paradigms are as important as the code you right. They are your tools and become more powerful when used correctly.
We’ve been travelling around turkey for the past three weeks and have relied heavily on bus travel to get around. Travel books have great info, but there are a couple of more things to think about when dealing with buses in Turkey.
First, there are several companies that serve various routes. Pamukkale, KamilKoc and Metro are the big ones. There are several more depending on where you are and where you’re going. If you don’t see the bus time you want, or if a bus is full, check with another company. Prices are fairly set so I don’t think it’s worth negotiating down. If you are booking through a tour operator, hotel or another reseller they will most likely book through another company, most likely one of the above.
It’s important to ask what type of bus you’ll be taking. There are big coach buses, older buses, and minibuses. Ideally you want to be on a big coach bus, often referred to as a big bus. There is usually wifi and tv (turkish only) on big buses, but I haven’t been on one yet with power. One did have USB outlets but was unable to charge the iPad. Minibuses and older buses may not have air conditioning, so it’s important to ask. Big buses have the smoothest ride and the most legroom. If you don’t like the bus you’re getting at the time you want, see if there’s another time with a better bus or go to another company. Always get your ticket from someone behind a desk. There will be plenty of people trying to sherpa you here and there, but just go right to the desk. At some otogars there are valets to help you. They may appear to be trying to sell you something. Just ask the right questions and you’ll be fine. Turkish people are very nice and very helpful.
The bus may make a lot of stops. We were on a minibus from Denizli to Fethiye and the bus stopped for anybody along the road. It was nuts! People would be waiting on the road no more than 50 meters away from each other and the bus would pull up, slow down, see if anybody needed to get on. Also, some buses will stop at rest areas every 45 minutes to an hour for breaks. On our way to Selcuk we had to stop at a rest area for 15 minutes even though we were only five minutes out from our destination. Ask if it is a direct bus and how many stops it will make. Usually the big coaches are better than the minibuses in terms of stopping. If possible, just avoid minibuses. The ride will most likely be bumpy as well unless it is a newer minibus or a tourist minibus.
Seats are assigned on buses, so ask for a seat up in the front. Some bus companies have seat maps so you can see where you’ll be seated. You don’t need to rush onto the bus, just put your bags on, get on, and find your seat. Everyone is very nice and will gladly help you out. You’ll also get tea or coffee on the bus with a snack. If the bus is really bumpy don’t get anything hot. You’ll probably spill it, need to drink it really quickly, than have to go to the bathroom. Most buses don’t have bathrooms (they do make a lot of stops, so don’t worry, but some restrooms cost 1TL). Another funny thing is that on minibuses in small towns there will be guy walking up and down with lemon or rose oil for your hands. A nice little refresher!
Dolmuses are fantastic. These are little vans that go around towns to pick people up and drop them off along the way. They are extremely cheap, extremely frequent and should be leveraged. They are just as good as taxis and cost a lot less. They are great within cities to get to more remote areas and to travel among smaller towns. They are great on the Turqouise coast to explore different beaches. Essentially, you just wait outside on the road in the direction you want and a van with people will pull up. Hotel, pensyon and guest house operators are very helpful with Dolmus transport. Depending on where you are on the Lycian way you could even send your bags ahead to be picked up by your next stop.
If you stick with the big buses and know your options bus travel in Turkey is a great and economical way to get around. Always bring earplugs and an eyemask, especially on night buses. There will always be a crying baby and someone reading with the light on.
Caching is one of the most effective techniques to speed up a website and has become a staple of modern web architecture. Effective caching strategies will allow you to get the most out of your website, ease pressure on your database and offer a better experience for users. Yet as the old adage says caching–especially invalidation–is tricky. How to deal with dynamic pages, deciding what to cache, per-user personalization and invalidation are some of the challenges which come along with caching.
There a three broad levels of caching:
HTTP Caching allows for full-page caching via HTTP headers on URIs. This must be enabled on all static content and should be added to dynamic content when possible. It is the best form of caching, especially for dynamic pages, as you are serving generated html content and your application can effectively leverage reverse-proxies like Squid and Varnish. Mark Nottingham’s great overview on HTTP Caching is worth a read.
Fragment Caching allows you to cache page fragments or partial templates. When you cannot cache an entire http response, fragment caching is your next best bet. You can quickly assemble your pages from pre-generated html snippets. For a page involving disparate dynamic content you can build your result page from cached html fragments for each section. For listing pages, like search results, you can build the page from html fragments for each id and not regenerate markup. For detail pages you can separate less-volatile or common sections from high-volatile or per-user sections.
Object Caching allows you to cache a full object (as in a model or viewmodel). When you must generate html for each user/request, or when your objects are shared across various views, object caching can be extremely helpful. It allows you to better deal with expensive queries and lessen hits to your database.
The goal is to make your response times as fast as possible while lessening load. The more html (or data) you can push closer to the end-user the better. HTTP caching is better than fragment caching: you are ready to return the rendered page. When combined with a CDN even dynamic pages can be pushed to edge locations for faster response times. Fragment caching is better than object caching: you already have the rendered html to build the page. Object caching is better than a database call: you already have the cached query result or denormalized object for your view. The deeper you get in the stack (the closer to the datastore) the more options you have to vary the output. Consequently the more expensive and longer the operation will take.
A cache strategy is dependent on breaking content down to store and reuse later. The more granular you can get the more options you have to serve cached content. There are two main dimensions: what to cache and whom to cache for. It is difficult to HTTP cache a page with a “Hello, {{ username }}” in the header for all users. However if you break your users down into logged-in users and anonymous users you can easily HTTP cache your homepage for just anonymous users using the vary http-header and defer to fragment caching for logged-in users.
Cache key naming strategies allow you to vary the what with the who for in a robust way by creating multiple versions of the same resource. A cache key could include the role of the user and the page, such as role:page:fragement:id, as in _anon:widgetdetail:widget:1234 and serve the widget detail html fragment to anonymous users. The same widget could be represented in a search detail list via _anon:widgetsearch:widget:1234. When widget 1234 updates both keys are invalidated. Most people opt for object caching for an easy win with dynamic pages, specifically by caching via a primary key or id. This can be helpful, but if you break down your content into the what and who for with a good key naming strategy you can leverage fragment caching and save on rendering time.
The vary http header is very helpful for dealing with HTTP caching and is not used widely enough. By varying URIs based on certain headers (like authorization or a cookie value) you can cache different representations for the same resource in a similar way to creating multiple keys. Think of the cache key as the URI plus whatever is set in the vary header. This opens up the power of HTTP caching for dynamic or per-user content.
You are ready to deliver content quickly when you think about your cache in terms of views and not data. Cache a denormalized object with child associations for easy rendering without extra lookups. Store rendered html fragments for sections of a page that are common to users on otherwise specific content. “Popular” and “Recent” may be expensive queries; storing rendered html saves on processing time and can be injected into the main page. You can even reuse fragments across pages. A good cache key naming strategy allows for different representations of the same data which can easily be invalidated.
Nobody likes stale data. As you think about caching think about what circumstances to invalidate the cache. Time-based expirations are convenient but can usually be avoided by invalidating caches on create and update commands. A good cache key naming strategy helps. Web frameworks usually have a notion of “callbacks” to perform secondary actions when a primary action takes place. A set of fragment and object caches for a widget could be invalidated when a record is updated. If cache values are granular enough you could invalidate sections of a page, like blog comments, when a comment is added and not expire the entire blog post.
HTTP Etags provide a great mechanism for dealing with stale HTTP requests. Etags allow a more invalidation options than the basic if-modified-since headers. When dealing with Etags the most important thing is to avoid processing the entire request simply to generate the Etag to validate against (this saves network bandwidth but does not save processing time). Caching Etag values against URIs are a good way to see if an Etag is still valid to send the proper 304 NOT MODIFIED response as quickly as possible in the request cycle. Depending on your needs you can also cache sets of Etag values against URIs to handle various representations.
If you must rely on time-based expiration try to add expiration callbacks to keep the cache fresh, especially for expensive queries in high-load scenarios.
Edge Side Includes are a great way of pushing more dynamic content closer to users. ESIs essentially give you the benefits of fragment caching with the performance of HTTP caching. If you are considering using a tool like Squid or Varnish ESIs are essential and will allow you to add customized content to otherwise similar pages. The user panel in the header of a page is a classic example of an ESI usage. If the user panel is the only variant of an otherwise common page for all users, the common elements could be pulled from the reverse-proxy within milliseconds and the “Welcome, {{USER}}” injected dynamically as a fragment from the application server before sending everything to the client. This bypasses the application stack lightening load and decreasing processing time.
Distributed and/or centralized caches are better than in-memory application server cache stores. By using a distributed cache like Memcache, or a centralized cache store like Redis, you can drop duplicate data caches to make caching and invalidating objects easier. Even though caching objects in a web app’s memory space is convenient and reduces network i/o, it soon becomes impractical in a web farm. You do not want to build up caches per-server or steal memory space away from the web server. Nor do you want to have to hunt and gather objects across a farm to invalidate caches. If you do not want to support your own cache farm, there are plenty of SaaS services to deal with caching.
Compressing content helps. Memory is a far more valuable resource for web apps than cpu cycles. When possible, compress your serialized cache content. This lowers the memory footprint so you can put more stuff in cache, and lightens the transfer load (and time) between your cache server and application server. For HTTP caching the helpful vary http header can also be used to cache content for browsers supporting compression and those that don’t. For object caching, only store what you need in the cache. Even though compression helps reduce the footprint, not storing extraneous data further reduces the footprint and saves serialization time.
One of the interesting trends I am reading about is how certain NoSQL stores are eliminating the need for separate cache farms. NoSQL solutions are beneficial for a variety of reasons even though they create significant data-modeling challenges. What NoSQL solutions lack in the flexibility of representing and accessing data (i.e. no joins, minimal search) they can make up in their distributed nature, fault-tolerance, end access efficiency. When you model your data for your views, putting the burden on storing data in the same way you want to get it out, you’re essentially replacing your denormalized memory-caching tier with a more durable solution. Cassandra and other Dynamo/Bigtable type stores are key-value stores, similar to cache stores, with the value part offering some sort of structured data type (in the case of Cassandra, sorted lists via column families). MongoDb and Redis, (not Dynamo inspired) offer similar advantages; Redis’ sorted sets/sorted lists offer a variety of solutions for listing problems, MongoDb allows you to query objects.
If you are okay with storing (and updating) multiple-versions of your data (again, you are caching for views) you can cut the two-layer approach of separate cache and data stores. The trick is storing everything you need to render a view for a given key. Searches could be handled by a search-server like Solr or ElasticSearch; listing results could be handled by maintaining your own index via a sorted-list value via another key. When using Cassandra you’d get fast, masterless, and scalable persistant storage. In general this approach is only worthwhile if your views are well-defined. The worst thing you want to do is refactor your entire data model when your views change!
There is always debate on differences between frameworks and languages. One of the things I always look for is how easy it is to add caching to your application. Rails offers great support for caching, and the Caching with Rails guide is worth a read no matter what framework or language you use. It easily supports fragment caching in views via content blocks, behind-the-scene action caching support, has a pluggable cache framework to use different stores, and most importantly has an extremely flexible invalidation framework via model observers and cache sweepers. When choosing any type of framework, “how to cache” should be a bullet point at the top of the list.
I wrote a post for the Getty Images Technology blog on how we use SOA at Getty Images. As systems become more and more complex with unique scalability needs, SOA allows systems to segment complexity and create useful boundaries.
http://blog.gettyimages.com/2012/02/21/put-a-service-on-it-how-web-services-power-getty-images
In a comment to my earlier article Thoughts on Kanban someone brought up the subject of end dates. Businesses obsess about the “When can we have it?” question. Dates and deadlines trump all. Let me tell you a secret: dates are bullshit. It is a prohibitive mentality in today’s world. Technology needs to reframe the question. Stakeholders need to change their engagement. No company ever succeeded because they made dates. Companies succeed when they continuously deliver innovation. It is not about the destination. It is about the journey and where you end up.
For agile to truly succeed the DNA of the company–top to bottom–must be continuous improvement through continuous delivery.
A friend told me a story of a prioritization meeting he had with a stakeholder. After fleshing out seven distinct features with the dev team, the stakeholder was asked to prioritize. He walked up to the board, put a “1” next to everything, and walked out.
Really? Everything is a top priority? So you are saying you would rather have nothing than anything? Then what you want is worthless. You may think you need everything but you are showing your unwillingness to change or improve. People that cannot work through small changes definitely cannot deal with large ones.
Yes, long-term vision is important. It is the goal. It ensures that everyone heads in the right direction every step of the way. It helps people make reasonable decisions. But it is still long-term; it is just a vision of the future negating the small details that allow the day-to-day. It is painful, ridiculous and unnecessary to wait for the future to just “appear”.
Long term deliverables create impatient, anxious users. It creates large, unmanageable codebases, complex releases, excessive bugs. It disconnects original vision from delivered functionality. The cherry on the cake: it creates a confused user base making awkward and painful adjustments to radical new processes. There is no long-term goal that cannot be broken down and reached via small iterative releases. Baby steps. One at a time, together. It is a three-legged race for everyone.
This is the root cause of scope creep. Okay, we’re working on feature x, but can we do this? Can we do this? What about this? You tell me. Is it more important for you to do that or get what we have out? It is your call! When you look at the backlog is it more important to enhance the current feature or move to the next thing on the list?
We have daily scrums to answer the improve or move question. Get involved in the day-to-day. Transparency is king. If you don’t trust the person making the call then don’t let them make the call. Empower key people: it’s an organizational change which pushes agile forward.
Active engagement between stakeholders and developers is agile development. The less barriers between the two the better. Getting a common sub-conscience understanding of “what we are doing” is the key to success. Small companies align on vision easily. Large companies need to break up into teams and align on goals.
You want to know if this really complex thing you are asking for can be done in eight weeks? Can you spell out every possible detail, define every wireframe, tell me how it should look on every device to every user in the world, outline every workflow, specify the amount of load it requires, explain how you will want to enhance it in the future, not bother us at all while we build it, let us decide any confusing or ambiguous detail, then maybe, maybe, I can do this thing that nobody has ever done before in eight weeks. If eight weeks later when you are unhappy with that one thing you did not explicitly specify (even though I totally asked you to specify everything) I will tell you “It wasn’t in the reqs”.
Is that how you want to work? Or would you rather tell me the gist of what we are doing, come up with a plan to get there, see what we can do first quickly, get it out, then take it from there. Is outlining every validation error on every page necessary to do now or can we start with the first page and take it from there?
You may also use deadlines to motivate people. Vision, direction and importance also motivate people. It is pretty easy to get something done by saying “This needs to happen by this date”. But that shows you do not care what it does or how well it works. You are asking people to time box something because either the details are irrelevant or there is no trust in people to make the right decisions.
Releasing functionality does not mean users have to see it. Turning features on and off, experimenting with small audiences, refactoring one class rather than an entire stack; all these are powerful steps for modern companies to improve products. Constant feedback lets everyone know they are headed in the right direction. It lets dev teams know the health of their code base. What is better than actually integrating new code to production to test integration? What is better than knowing if a feature is worth investing in than testing it on a small set of production users?
I’m sure you heard about the cone of uncertainty. Small and explicit features with short estimations can be delivered accurately. Larger loosely defined features with long dates are difficult to predict. Break down large features into small, clear user stories. A big feature or a long date means you do not care about details.
Short date ranges are okay and can help coordinate people. They work well when matched with story size. A few days, one week, one to three, and three to five are good ranges which require a decent discussion to work out details. Ranges allow for adjustment and can be refined as you move along the uncertainty cone. Ideally they are auto-calculated from story points. Anything +5 weeks requires a break down; there are too many variables. Don’t think you can add up ranges either, that is not the way it works. It is about increasingly clarifying level of detail on what’s ahead to maintain momentum. You can still, and should, deploy intermittently within the date range. Long dates don’t provide detail. Without detail you do not care how features work. So you do not care what you get.
Your type of business does not matter. Your size does not matter. In house, off shore, outsourced development does not matter. It’s 2012. Your company uses technology to do business. You work for a tech company. As a tech person your job is to help your business realize this. As a stakeholder your job is to realize this and help your tech team help you do your job faster, better, easier.
Good companies consistently take their products to the next level. How? They build an incredible manufacturing pipeline. Why is Toyota’s just-in-time practices so applicable to building software? Because development teams manufacture software. It’s how the product is built. It’s how it changes. It’s how it’s delivered. It’s how it’s fixed. Dev teams buy the land, construct the building, build the robots, define the pipeline, assemble the pieces, run quality control, load up the trucks, deliver and when all that is done they improve. Hopefully the new manufacturing plant allows for easy improvement. Otherwise somebody made a mistake.
It is everyone’s responsibility to ensure that manufacturing pipeline delivers as efficiently as possible with no flaws. Continuous integration, unit tests, programming languages, server frameworks, agile development, clear vision, well written stories, cohesive vision, user feedback; it all goes into building a solid manufacturing process. You do not need to get it right the first time. You just need to change and improve when needed. Tech leaders ensure they are building the right process for the business. Stakeholders enable and leverage that pipeline effectively. Radical product changes, disconnected vision or tech decisions lead to numerous and slow manufacturing plants.
Dates are bullshit. It’s about where you are, where you want to be, and what’s next. That’s the conversation.