The Road Less Traveled

When there are two paths to your destination, why does it always seem like you’re on the road with more traffic?

Olena Shmahalo/Quanta Magazine

Two roads diverged in a wood, and I—
I took the one less traveled by,
And that has made all the difference.
—Robert Frost

On the road, I always try to follow the example of Robert Frost, and I am sure you do too. Every time I drive, I look for the road or lane less traveled. Almost invariably, I find myself on the road more crowded, doomed to fritter away precious time while the cars on the other side whiz by — or so it seems. Why does the road not taken always seem less crowded? Why is this frustration so universal? It’s a puzzle both mathematics and psychology claim to explain, so let’s explore it with some puzzles of our own.

The mathematical or statistical explanation is called selection bias. It’s a simple idea: The busier road has more drivers on it, so if you sample a bunch of drivers randomly, more of them will be from the more crowded road. How large is this effect? Let’s find out in our first problem.

Question 1: If I want to drive from my home in Connecticut to the Quanta Magazine offices in New York, I have two perfectly good roads to follow: the interstate highway I-95, or the picturesque Merritt Parkway. Let us say the drivers of 200 cars independently and randomly make their choice between these two roads with a 50 percent probability of choosing a given road. Assume that there are no other cars on the road. How many more of the 200 cars end up on the more crowded road? (Update: The solution is now available here.)

It’s not an easy calculation, and if you can’t do it, make a guess. The exact number is not important — this problem introduces the neat puzzle that follows. There are a significant number of extra cars on one of the roads, just by chance, even if the probability of taking either road is the same. And because there are more cars, you and other drivers are more likely to be among them. Can this really be the cause of road-choice frustration?

A monthly puzzle celebrating the sudden insights and unexpected twists of scientific problem solving. Your guide is Pradeep Mutalik, a medical research scientist at the Yale Center for Medical Informatics and a lifelong puzzle enthusiast.

Perhaps, but psychologists point to different explanations — the effect of emotional arousal on memory, followed by confirmation bias. We normally expect to have a smooth ride, so if the traffic is obliging, we don’t react emotionally — it’s what we expected all along. But if we are stuck in traffic on a crowded road, we do have a strong emotional reaction, and we remember the incident much more vividly. It colors our outlook disproportionately, and we start to feel that we always take the busier road. This feeling is reinforced every time this happens, as it will from time to time by chance, and these incidents just “confirm” our suspicions that the world is stacked against us. This is the psychological explanation not just for driver’s frustration, but for a whole host of eponymous laws related to frustration such as Murphy’s law — “If anything can go wrong, it will” — and my favorite, “Fetridge’s law,” which states that “important things that are supposed to happen do not happen, especially when people are looking.” This last one affects the parents of little children quite a lot!

So is it all in our heads? Let’s turn back to mathematics. The calculation of the first problem is not the whole mathematical story. Even though a particular driver is more likely to end up on the busier road, there are times when you will be on the road less traveled. Doesn’t that decrease your tendency to overestimate, or engage in upward bias, somewhat?

Let us try to estimate this bias by imagining that there are just two cars choosing one of the two roads at a given time. In this case there is a 25 percent probability that both cars take I-95, a 50 percent probability that there is one car on each road, and a 25 percent probability that both cars take the Merritt. So if you interview the drivers and ask them how many cars they saw, including their own, the expected number of cars for each driver on his or her chosen road comes out to be 0.25 x 2 + 0.5 x 1 + 0.25 x 2 = 1.5. But this is wrong, because if you “interview the roads,” which is the right way to do it, we can see that each road has a total of 0.25 x 2 + 0.5 x 1 = 1 car. So each driver has a perceived upward bias of 50 percent. That’s quite impressive. Is this bias causing our problems?

Question 2: What happens to the driver’s upward bias if there are an average of 100 cars on each road? How many cars do we think he or she sees?

The answer to this problem can be reached by doing intricate calculations, but you can also figure it out by thinking simply, but deeply. The answer may surprise you. If you figure it out, explain how you reached your conclusion.

So after all this, what do you think is the source of “other lane” or “other road” envy? Can selection bias still cause this feeling, but in some other unexplored way? Maybe, as we saw in the last puzzle, our mathematical models are merely scratching the surface of a complex phenomenon and need to be deepened. Certainly, drivers don’t make their decisions independently and there is some form of group dynamics that we need to model. But maybe it’s all in our heads, as the psychologists tell us. Or maybe the universe is really stacked against us. It’s something to ponder the next time you are stuck in bumper-to-bumper traffic. Do send us the results of your cogitations in the comments below.

Happy puzzling. And may the insight be with you!

Editor’s note: The reader who submits the most interesting, creative or insightful solution (as judged by the columnist) in the comments section will receive a Quanta Magazine T-shirt. And if you’d like to suggest a favorite puzzle for a future Insights column, submit it as a comment below, clearly marked “NEW PUZZLE SUGGESTION” (it will not appear online, so solutions to the puzzle above should be submitted separately). Update: The solution has been published here.

View Reader Comments (19)

Leave a Comment

Reader CommentsLeave a Comment

  • Well, I'm going to go with the obvious answer, which is that he should expect to see 101 cars. The only difference between the Merritt Parkway's point of view and the hypothetical driver's point of view is that the Merritt Parkway doesn't count the observer when the observer takes I-95, but the observer always counts himself, no matter what road he takes. And since the Merritt Parkway should have an unbiased expected value estimate of 100.5 drivers (201 drivers, each of whom is 50% likely to take the Merritt), the observer should expect 101 drivers, equal to the unbiased estimate plus the extra 50% of the time he counts himself.

  • Q1: about 11.27 cars, on average.

    Q2: only half a car! The number of cars in your lane is 1 (you) plus on average half a car from each of the remaining T-1 drivers where T is the total traffic. That's T/2 + 1/2 on average, or just 0.5 greater than the unbiased mean.

  • Here is one way to solve the puzzle:
    If we exclude ourselves, then each road has on average 99.5 cars (because without us, there are 199 cars, so 199/2 cars per road). Therefore, each person sees 100.5 cars on average (the 99.5 plus itself).

  • For question 1, we can start with the difference in the number of cars on road A and road B. If there are n total cars and the number on road A is k, then the number on road B is n-k and the difference is 2k-n. The difference in cars on the crowded road and the less-traveled road is the absolute value 2k-n or |2k-n|. Given that the distribution of cars on road A (or B) is a binomial distribution with p = 1/2, the mean value of |2k-n| is approximately 2 sqrt(n/2pi). For 200 cars that is about 11.3 more cars on the more crowded road.

    Question 2 is much simpler. Assuming that I am one of the 200 drivers and each driver chooses a road independently, then there are on average 199/2 other cars on the other road, and the same number on the road I choose, plus my car. So there is an average of one more car on the road I choose (my own car, that is), and this is true no matter how many total cars are on the road.

  • You're kind of jumping all in on that Robert Frost poem, you should know it doesn't imply that it was necesarily *good* to have taken the road less traveled.

  • I have heard a different explanation for cars traveling in neighboring lanes. For one thing, in this problem many people can and do change lanes, so any imbalances can be equaled out. But it still seems like you are going slower than the other lane. Suppose you take turns passing as many cars as pass you. When you pass five cars they are going slow (perhaps not moving) and they are closer together than when those same five cars are going faster and are passing you. It looks like a longer distance of cars passes you as compared to the distance of cars you pass when you are going fast.

  • The simplest solution to this issue lies not with Murphy nor Fetridge, but rather with Blaise Pascal, who wrote that "all of humanity's problems stem from man's inability to sit quietly in a room alone." If we all stayed at home where we belong, there would be no traffic jams.

  • A different take on the solution, is that we spend more time in the slow lane than the fast lane (I mainly think about this in the context of check out lanes, or TSA screening lanes). Since I spend more time in the slow lane than the fast one, I have more time to form a memory of the slow one than the fast one. So even I actually do pick the fast lane half the time and the slow lane half the time, I still spend more time in the slow lane, and when I'm in the fast one, it doesn't occur to me that I could be way the heck back there.

    As for a math answer to the 100 cars on two roads question, I get the following:

    First, I note that there are 2^N possible arrangements. Next, assume that I am in the left lane, so we will have to multiply the answer by two. There is one way for me to be by myself, (N-1, k) ways for me to have to k additional cars on my lane (not including myself) where (n, k) is n choose k. Then we want to evaluate the sum of (k+1)*(N-1, k) from k=0 to N-1. Note that the (k+1) term is the number of cars in the lane, so it is the weighting of the sum, and the binomial term is the number of times that combination can appear. Then, we sum that, multiply by 2 (I could be in either lane) then divide by 2^N (possible configurations). The sum is evaluated to (N+1)2^(N-1) (see the wiki page form binomial coefficients for how to do these easy sums). Then we divide by 2^N to get (N+1)/2. For N=100, this is 50.5 cars. So the bias is only a half a car on average. For N=2, this bias is large (50%). For N=100, it is only 1%.

    And let's be honest, if there were only two cars on the highway, traffic wouldn't be a worry no matter what!

  • The reasoning given by eJ, Enrique Treviño, and Tom Nichols is correct (and RobertB as well, adjusting for the misunderstanding of 200 drivers other than oneself, for a total of 201 rather than the specified 200 total).
    For fun, here is a one-liner in python for question 1 (using scipy.misc.comb(n,k) for the binomial coefficient and with n=200):
    In [3]: sum([abs(n-2*k)*comb(n,k)/2**n for k in range(0,n+1)])
    Out[3]: 11.2696958
    consistent with eJ and Tom Nichols

  • p.s. re
    > In [3]: sum([abs(n-2*k)*comb(n,k)/2**n for k in range(0,n+1)])
    Had meant to mention that with a line or two of simple algebra the above sum evaluates exactly to
    n*{n\choose n/2}/2^n = 200*(200!/100!^2)/2^{200} = 11.2696958…
    (for n even).
    Tom Nichols' estimate of 2 sqrt(n/2pi) comes from the large n behavior of {n\choose n/2}, by Stirling's approximation.

    Equivalently, for n=200 and p=1/2, the binomial distribution becomes close enough to a normal (gaussian) distribution with variance sigma^2= n*p*(1-p)=200*(1/2)*(1/2)=50. The expectation value of a half-gaussian with standard deviation sigma is 2*sigma/sqrt(2*pi)= 5.64, so the expected difference between the higher and lower roads is twice that, or roughly 11.28 .

  • This puzzle takes us into the tricky realm of conditional probability.

    For Question 1, the task is straight forward, if tedious. Let's first label the roads A and B, and consider n total cars. First, let us ask "what is the probability that road A has exactly k cars?" This is given by the well-known binomial distribution: P[k cars] = nCk*p^k*(1-p)^(1-k), where nCk is the "choose" function from combinatorics: nCk = n!/(k!*(n-k)!), and p is the probability that each car will, independently, decide to drive on road A. Since we take the roads to be otherwise equivalent, we have p=0.5, so the binomial distribution reduces to P[k cars] = nCk/2^n. A quick check confirms that the expectation value of this distribution is E[cars] = p*n = n/2.

    Now we want to ask "how many cars, on average, are on the more crowded road?" If there are k cars on the more crowded road, then the number of cars on the more crowded road is given by max(k,n-k). The average is, therefore, the expectation value E[more crowded] = sum(k from 0 to n) max(k,n-k)*P[k] = 0.5^n*sum(k from 0 to n) max(k,n-k) nCk. This is straight forward but tedious to compute; for large values of n you will need a computer.

    However, a more interesting case (Question 2 above) to compute is "what percentage of time am *I* on the more crowded road?" Naively, one might reason this way: if k cars are on road A, then the likelihood that I am on of them is k/n, so therefore the likelihood that I am on the more crowded road is max(k,n-k)/n. However, this is incorrect, because my choice of which road to drive on is independent of the distribution of the rest of the cars – but the probability that k cars are on road A is /not/ independent of my choice.

    So, let's say (without loss of generality) that I choose to drive on road A. What percentage of the time will road A be more crowded, given that I choose to drive on it? Let k' be the number of cars excluding myself on road A, and let n' = n-1 be the total number of cars excluding myself. Then road A is more crowded whenever k' + 1 >= n/2, or k' >= (n'-1)/2. This total probability is the sum of each such case; P[A is more crowded] = sum(k' from (n'-1)/2 to n') n'Ck' / 2^n'. Some quick arithmetic will show that, when n = 2 (myself and one other car), P = 1. For n=3, P = 0.75. n = 4, p = 7/8; n = 5, p = 11/16; etc. (Note that the saw-toothed pattern is due to the presence of ties.) In generally, p shrinks as n grows, with the limit of p = 0.5 when n goes to positive infinity.

    The conclusion here is that, when the number of cars is small, my presence on road A significantly affects the distribution of cars; however, as the number of cars grows, my presence becomes negligible, and the likelihood that I will be on the more crowded road approaches 50%.

    So, what is going on here? It's obviously *not* the case that random chance is responsible for the perception that we always wind up on the more crowded road. While it's true that confirmation bias comes into play here, I'd say that's not the real culprit either. The actual answer is this: when we are traveling on road A, we (generally speaking) don't get to observe the number of cars traveling on road B. We only know k, without knowing n. So we can't actually know if road A is "more crowded" or "less crowded" – we only know that road A is crowded. So, we have to rely on some sort of Bayesian prior expectation for the number of cars on road B, based on previous times we've driven on it. If both roads were truly equal (and we had sampled them both equally well), we'd estimate that on average there were about n/2 cars on road B at any given time, so we'd guess that road A is more crowded about half the time. Unfortunately, humans are very, very bad at this kind of estimation. Here, another human bias comes into play; one colloquially known by the phrase "the grass is always greener on the other side of the fence."

    And so, I shall be telling this with a sigh, somewhere ages and ages hence. Two roads diverged in the city, and I – I took the one more crowded by; and that has made not one jot of difference.

  • My guess is that 20 to 40 cars will take the Merritt Parkway, and most others i.e. 180 to 160 will take the I-95. My premise is not just the general 80-20 rule approximation. I assume wealth distribution is skewed across these as a uniform representation of Americans and that 1% will be chauffeured and likely take the fastest route based on traffic condition information so on. Another 9% will be those smart, highly self respecting folks who will also choose time over distance, have risk taking in them and may go to Merritt. The rest 90% will be wavering or perhaps more relaxed, listening to radio or their favourite radio channel or just thinking nothing and dont really care who will head towards I-95. However from this 90% there are bench sitters who aspirationally want to be like the successful, risk taking (not gambling), time efficient, productive folks and in that "wait a minute moment" will take the Merritt Parkway. These represent the additional 10% range give and take 2-3 points. To make up for this fluctuation, there will be a few from the 9% smarties who are inconsistent, being human naturally, and a 2-3 point variation, that sort of balances out the momentary achievers. Hope you enjoyed. Cheers!

  • One thing to consider in the multi-lane case is that in slow traffic, the ability to move comes in waves that are usually synchronized across lanes. Sometimes, though, particularly when a sudden wave with high amplitude (unusual max speed) comes by, it can hit different lanes at noticeably different times. So, if it hits the lane next to you first, you might change lanes to catch it, only to find yourself stop and watch the lane you were in catch the wave. Further, after you move into the fast lane and bog it down, that action can accentuate the amplitude of the wave, increasing the chance of the exact same thing happening down the line.

    There is a psychological side to this description as well: when the wave hits your lane first, it is easier to accept that the lane next to you gets some motion too, since you just passed them up so much. But when the other lane gets it first, it just feels unfair because there's no indication you'll get benefit in your lane in the future.

  • Another way to look at this: from one road a driver cannot see the other road, so the comparison he makes is not with the journeys of drivers on the other road at that same time, but with other journeys he himself has made previously (on either road), so the more crowded the road he has chosen on "this" particular journey the more unfavourable will be his comparison with past journeys he has made, and the more he will feel that whenever traffic is heavy he makes the wrong choice so the world is out to get him. The answer of course is to create a win-win, which means take Lutz Barz's advice, travel by bus, and take a good book. Quick journey = quick arrival = happiness. Slow journey = good read = happiness.

  • A more interesting question is how many times we curse our luck for being on the more crowded road. For 4 people, the probability of having a more crowded road is 5/8 (1/16 + 1/16 + 4/16 + 4/16 for 4,0; 0,4; 3,1; 1,3 cars respectively) and the probability of one being on the more crowded road is 4/8. So when a crowded road occurs, there is a 4/5 chance of one being on the more crowded road simply because of the fact that it is people like us causing the more crowding in the first place.

    This is true for queues at counters and other places too.

  • At the super(stitious) market, no matter which line I stand in, something goes wrong with that line. Commonly, in the line ahead of me, there will vintage (old) people, having too much vanity to use their glasses but insisting on counting a month's worth of collected change to pay for their items – and then discovering they are 4 cents short – but then, with miraculous and pious hope, will faithfully and very meticulously re-excavate their handbag(s)/keycase/wallet to unsuccessfully seek aforesaid 4 cents. Then they and/or the following customer will attempt to pay with a recalcitrant credit card.
    Of course everyone in that stopped line knows that the fault and blame lie with me because I voluntarily joined that line……….
    If I now move to a fast moving line, it will promptly stop, (no paper in the till compounded by an almost military incompetence in installing a new roll of paper etc) so both lines are now angry with me. (sigh)
    Thankfully the voices in my head reassure me that it's not completely my fault and it is partly due to my incredibly fantastic unique exceptionalism and I also gain comfort from the sawed off AK47 I always keep in the one leg of my pants.
    The only occasions that Murphy (who is/was actually a very optimistic legislator) proves to be wrong, is when one relies on his pronouncements to be accurately correct!
    Where's my T shirt?

  • ANSWER: Perspective!
    Perspective increases the value of myself or my “offenders”.
    Positive perspective allows me to see the road less traveled as, well, less traveled. Negative perspective gives the ‘value’ to the others who are there to offend me and so the less traveled road is filled with jerks!
    By concentrating or committing to concentrate on the brighter side of what we see (or don’t see) we actually change our reality! After a while, all your life seems sunny and bright!

  • There is also a time based selection bias. If a person drives on 5 1km roads and the first 4 take him 5 minutes and in the last he's stuck in traffic and it takes him 15, that's 15/35 or 43% of his journey but only 20% of his distance. So 43% of the time he's stuck in a jam even though it happens on only 20% of roads.

Comments are closed.