Tag Archive for 'latency'

Performance Basics: Latency, Throughput and Load

I am often surprised by the fact that sometimes even seasoned IT professionals get confused by the differences between latency, throughput (aka bandwidth) and load. Simply put, here are the differences:

  • latency is the time it takes for a single operation to complete. For a client accessing a website for instance, this means “the time it takes to load one page”. In the web world, latency will be more often called “response time”.
  • throughput is the number of operations a system can deliver per unit of time. To take the website example again, this means “how many pages can be served per second”
  • the load can be a bit of a fuzzy concept. Simply put it is about the relationship between the throughput and the latency. To take the webserver example again, this means: “How many pages per second can I deliver with each page being served under xxx milliseconds?”

To take a plumbing analogy, the latency is determined by the length of a pipe and the throughput is determined by the size of the same pipe.

Depending on the need of users, throughput might be more important than latency or vice versa:

  • In the case of online games, latency is often more important since clients are only transmitting small amounts of data (the position of the player).  The faster this small amount of data gets transmitted to the server the better: this can give you a competitive advantage. Let’s not forget that latency on a global planetary scale is dictated by distances as you will be limited by the speed of light and network equipments you have to go through, each adding a bit of latency: a transatlantic transfer of data, however small will take about 100ms. Interestingly enough, traders might face issues similar to online action gamers if the latency to reach a trading server is too high: they will have a competitive disadvantage.
  • In the case of video streaming, latency is not so much of an issue (it dictates how much you have to wait before the video starts) but throughput will determine the quality of the video (forget about HD content on a small pipe)

That is just a few examples but often, a given performance goal has to be defined in terms of latency and throughput as these are two very different things…

Next on the topic of Performance, we will be talking about bottlenecks.