Data crunching to find the cheapest airline in the world

Michael Cameron:

The growth of big data and availability of APIs is providing exciting new opportunities for making sense of travel data, even for a fledgling start-up like Rome2rio.

Airfares fluctuate wildly, but do follow certain obvious trends; longer flights cost more, and some airlines are more expensive per mile flown than others.

We recently started an internal project aiming to model approximate/typical air fares for the flight itineraries assembled by our system. Our aim was to use this model to improve the accuracy of our multi-modal routing engine. However, in the process we generated some interesting data worth sharing with the industry.

We modeled airfares using some simple parameters. To do this, we examined the economy class airfares displayed by Rome2rio to users over the past 4 months, totalling some 1,780,832 price points. We grouped the airfares by distance and selected the 20th percentile fare for each distance (where 20% of fares are less, and 80% are more), to produce the following graph:

“Bin Laden won, with our assistance. Our applause shows the scale of his victory.”

Fabius Maximus:

9-11 changed the course of a great nation, turning America decisively toward the dark side. Massive internal surveillance, militarization of police, endless war, hatred of Islam., torture, lifetime detention without trial, incessant propaganda, and a stream of fake terror plots (created by the government).

We pay for this with larger deficits, loss of global leadership, and corruption of our people (eg, jingoism, bloodlust). We see celebrate these things, the death of the America-that-once-was, by applauding the film “Zero Dark Thirty”.

Welcome to The New America! Brought to you by al Qaeda and the US government, with the willing assistance of the US people.

Interviews: Eugene Kaspersky Answers Your Questions

Last week, you asked questions of Eugene Kaspersky; below, find his answers on a range of topics, from the relationship of malware makers to malware hunters, to Kasperky Labs’ relationship to the Putin government, as well as whitelisting vs. signature-based detection, Internet ID schemes, and the SCADA-specific operating system Kaspersky is working on. Spoiler: There are a lot of interesting facts here, as well as some teases.

Taking the Positive Route: A look at Packer Coach Mike McCarthy, with a Bit of Corinthians

Lori Nickel:

When the Green Bay Packers suffered an embarrassing loss to the New York Giants in which they looked overmatched and outplayed, it could have demoralized a team with too many starters and stars on the sidelines in casts and on crutches.

Three days later, almost everyone expected a verbal lambasting from coach Mike McCarthy.

“But he gave this speech,” said nose tackle B.J. Raji, “and it just threw me.”

McCarthy quoted Corinthians from the Bible: “My grace is sufficient for you, for my power is made perfect in weakness.”

“The room was in complete silence,” said Raji. “It was not a long speech. I was surprised. And shocked. And kind of impressed. Losing a four-touchdown game on prime time, that’s cause for a fire-and-brimstone type of speech. It just shows how much he understands the players he’s coaching. I think the message got across.”

The message was to learn from the loss and convert it into something useful and therefore something meaningful.

You’ll never hear McCarthy say he’s out to prove the doubters wrong. You’ll never hear him vent his defenses to critics. One of his survival tactics is that he doesn’t waste his energy on anything he perceives as negative.

“I feel the ability to grow the positive is the best way to accomplish things,” McCarthy said last week in a one-on-one interview.


Nassim Nicholas Taleb:

The point we will be making here is that logically, neither trial and error nor “chance” and serendipity can be behind the gains in technology and empirical science attributed to them. By definition chance cannot lead to long term gains (it would no longer be chance); trial and error cannot be unconditionally effective: errors cause planes to crash, buildings to collapse, and knowledge to regress.

Something central, very central, is missing in historical accounts of scientific and technological discovery. The discourse and controversies focus on the role of luck as opposed to teleological programs (from telos, “aim”), that is, ones that rely on pre-set direction from formal science. This is a faux-debate: luck cannot lead to formal research policies; one cannot systematize, formalize, and program randomness. The driver is neither luck nor direction, but must be in the asymmetry (or convexity) of payoffs, a simple mathematical property that has lied hidden from the discourse, and the understanding of which can lead to precise research principles and protocols.


The luck versus knowledge story is as follows. Ironically, we have vastly more evidence for results linked to luck than to those coming from the teleological, outside physics—even after discounting for the sensationalism. In some opaque and nonlinear fields, like medicine or engineering, the teleological exceptions are in the minority, such as a small number of designer drugs. This makes us live in the contradiction that we largely got here to where we are thanks to undirected chance, but we build research programs going forward based on direction and narratives. And, what is worse, we are fully conscious of the inconsistency.

Corruption perceptions Index 2012

Looking at the Corruption Perceptions Index 2012, it’s clear that corruption is a major threat facing humanity. Corruption destroys lives and communities, and undermines countries and institutions. It generates popular anger that threatens to further destabilise societies and exacerbate violent conflicts.

The Corruption Perceptions Index scores countries on a scale from 0 (highly corrupt) to 100 (very clean). While no country has a perfect score, two-thirds of countries score below 50, indicating a serious corruption problem.

Peter Thiel And Rene Girard

Nabeel Qurishi:

“I suspect that when the history of the 21st century is written circa 2100, he [Girard] will be seen as one of the great intellectuals” – Peter Thiel

When Blake Masters was posting his great notes on Peter Thiel’s lectures at Stanford, I found myself fascinated by the influence of Rene Girard.

Girard’s an original thinker. I’m not fully persuaded by his worldview: it explains a lot, but not everything. I’ve summarized his views here.

1. Mimesis determines what you want: In Girard’s view, people have appetites, which are your basic evolutionary needs: e.g. hunger; and desires, which are all other wants, e.g. the desire for a diamond ring. Girard’s belief is that people form desires based on what others around them want.

This is known as the ‘mimetic mechanism’. People take their cues from the people around them. They use other people as ‘models’, and (subconsciously) want what other people want, while rationalizing the whole time. In the diamond ring example, companies like DeBeers create a want artificially, and it catches on like a virus. People want diamond rings because other people want them, but they rationalise it by saying “it shows that my partner loves me”.

Ethereal Macro Photos of Snowflakes in the Moments Before They Disappear

Michael Zhang:

Russian photographer Andrew Osokin is a master of winter macro photography. His photo collection is chock full of gorgeous super-close-up photographs of insects, flowers, snow, and frost. Among his most impressive shots are photographs of individual snowflakes that have fallen upon the ground and are in the process of melting away. The shots are so detailed and so perfectly framed that you might suspect them of being computer-generated fabrications.

They’re not though. The images were all captured using a Nikon D80 or Nikon D90 DSLR and a 60mm or 90mm macro lens.

How to Win at Forecasting

Daniel Kahenman:

The question becomes, is it possible to set up a system for learning from history that’s not simply programmed to avoid the most recent mistake in a very simple, mechanistic fashion? Is it possible to set up a system for learning from history that actually learns in our sophisticated way that manages to bring down both false positive and false negatives to some degree? That’s a big question mark.

Nobody has really systematically addressed that question until IARPA, the Intelligence Advanced Research Projects Agency, sponsored this particular project, which is very, very ambitious in scale. It’s an attempt to address the question of whether you can push political forecasting closer to what philosophers might call an optimal forecasting frontier. That an optimal forecasting frontier is a frontier along which you just can’t get any better.

PHILIP E. TETLOCK is Annenberg University Professor at the University of Pennsylvania (School of Arts and Sciences and Wharton School). He is author of Expert Political Judgment: How Good Is It? How Can We Know? which describes a twenty-year study in which 284 experts in many fields, including government officials, professors, and journalists and ranging from Marxists to free-marketeers, were asked to make 28,000 predictions about the future. He found they were only slightly more accurate than chance, and worse than simple extrapolation algorithms. The book has received many awards, including the 2006 Woodrow Wilson Award from the American Political Science Association and the 2008 Grawemeyer Award for Ideas Improving World Order.