It begins.
« previous post | next post »
Geoff Pullum to me:
What the hell is going on with all the years except 1920 being labeled "11 PM"? [link]
Me to Geoff Pullum:
You mean on the x axis? I'm not seeing it — must be a special feature for Scottish readers.
Geoff Pullum to me:
Yep. 1920 11 PM 11 PM 11 PM 11 PM 11 PM 11 PM 11 PM 11 PM 11 PM on the x axis. Corresponds to decades. It's not Scots dialect: 1930 is known here as "1930".
Me to Geoff Pullum:
Screenshot?
Geoff Pullum to me:
Me to Geoff Pullum:
The same link gets me this:
Something special about you or your environment, apparently…
Geoff Pullum to me:
This is what we call BLAMING THE VICTIM.
Today's xkcd:
So this also works with random inexplicable software glitches, apparently.
cameron said,
March 16, 2016 @ 3:26 pm
Curious. Does Dr Pullum see the same effect with other start and end dates, or this this specific to 1920-2000?
Michael Chen said,
March 16, 2016 @ 3:38 pm
Daylight savings?
Amy de Buitléir said,
March 16, 2016 @ 3:40 pm
Here in Ireland, I also see the "11 PM"s!
Amy de Buitléir said,
March 16, 2016 @ 3:44 pm
I tried experimenting with different years, different search phrases, different smoothings, case-insensitivity on/off, but the "11 PM"s persist.
Keith said,
March 16, 2016 @ 3:48 pm
Here in France, I get the proper years… no 11PMs for me.
Amy de Buitléir said,
March 16, 2016 @ 3:49 pm
Interesting… I see the problem in "Iceweasel" (a version of Firefox), but in "Chromium" (a version of Google Chrome), the labels are correct.
The problem (or lack of it also shows up on the main ngrams page.
https://books.google.com/ngrams/info
Deborah Pickett said,
March 16, 2016 @ 3:55 pm
This smells of daylight saving time. A lot of software incorrectly assumes that there are always 86400 seconds in a calendar day, but in places where daylight saving is observed, one day is 82800 seconds long and one is 90000 seconds long. Repeatedly adding 86400 to a time (say, midnight) might not get you to the next midnight but only to 11 pm.
There's obviously more going on than just daylight saving, but it would be interesting to see if all of the people who observe this bug are in regions that (now) have daylight saving but didn't in 1920.
In my location (Melbourne, Australia) the X axis displays fine, but we have the distinction of starting the year in daylight saving time, so i would expect that you northerners would see something different.
David D. said,
March 16, 2016 @ 4:24 pm
I think Deborah Pickett may be onto something about daylight saving time, but perhaps not about assigning the problem to historical use of daylight savings time.
I suspect it's all about whether places have adjusted to daylight savings time this year. Daylight savings time began in the US last weekend but will not begin in the UK/Ireland until March 27.
Anyway I think it's definitely based on location. Works fine for me here in California but when I change my clock to London time I get the same error. Only in Firefox though — Safari works ok.
Rubrick said,
March 16, 2016 @ 4:26 pm
In the software biz, precisely this sort of interaction is played out thousands of times a day in bug reports. "Doesn't occur for me, must be something weird in your setup."
J. W. Brewer said,
March 16, 2016 @ 4:26 pm
Do we have to await a future post in order to get a speculative hypothesis accounting for the rise/fall of the "cherry pie" alternative?
Oskar Sigvardsson said,
March 16, 2016 @ 4:26 pm
I don't see the glitch, either, that's really freaky (Chrome, Sweden).
My first instinct as a programmer is to say that it's a localization issue. It's possible that the numbers are run through some library that localizes them based on browser information and geolocation, and that software it's glitched. That would explain why different results come from different countries or browsers. However, the page doesn't appear to be localized at all, even for me in Sweden the numbers all use English formatting instead of Swedish (periods instead of commas as decimal points, for instance).
Another (perhaps more likely) possibility is that time is the key. "1930" references time, as does "11 PM". A string like that is ambiguous, even if you already know that it has to do with time. It could refer to a year, but it could also refer to 7:30 PM on a 24-hour clock. It's possible some code ran "1930" through some sort of date/time-formatting library, and out came the wrong thing. Wouldn't explain why they're all 11 PM instead of 9:30 PM, 9:40 PM, etc.
Another possibility along those lines is that a time-formatting library expected the incoming data to be in a different format than it was. Feed 0xFFFFFFFF into a library expecting a standard unix timestamp, it'll say that that's December 31, 1969 at 23:59:59. However, if the library expects the timestamp to be an unsigned integer (which is not standard), it will think that that timestamp is February 7, 2106 at 6:28:16 (you would have to be a particularly bad programmer to make this particular mistake, but it is an illustration of an error that could happen). Similar problems happen if there's a confusion with endianness, causing the same data to look wildly different on machines with differing architecture.
Maybe the format of the date fed in was supposed to represent "1930", "1940", "1950", etc., but the library just interpreted that as 11 PM for some strange reason like this. That wouldn't explain why it's different for different people, though. It might be some combination of this with the localization issue, that the time-localization for particular regions or browser configurations works differently in the code.
These explanations seems exceedingly improbable, but it's an exceedingly improbably bug. And I've seen experienced bugs with far stranger explanations than these.
Rod Johnson said,
March 16, 2016 @ 4:43 pm
“I say violence is necessary. Violence is a part of America’s culture. It is as American as cherry pie. Americans taught the black people to be violent. We will use that violence to rid ourselves of oppression if necessary. We will be free, by any means necessary.”
–H. Rap Brown, July 27, 1967
Rod Johnson said,
March 16, 2016 @ 4:45 pm
Oops, should have linked that: https://books.google.com/ngrams/graph?content=American+as+cherry+pie%2CAmerican+as+apple+pie&year_start=1960&year_end=1980&corpus=15&smoothing=0&share=&direct_url=t1%3B%2CAmerican%20as%20cherry%20pie%3B%2Cc0%3B.t1%3B%2CAmerican%20as%20apple%20pie%3B%2Cc0
Theophylact said,
March 16, 2016 @ 4:59 pm
Yeah, that's why the peak. Brown was playing on the "apple pie" standard line.
Zizoz said,
March 16, 2016 @ 5:17 pm
My first thought was that "1930" probably actually represents "January 1, 1930 at 12:00 a.m." and that somehow as a result of Daylight Savings time, that became "December 31, 1929 at 11:00 p.m." which then gets displayed as "11 PM".
Electric Dragon said,
March 16, 2016 @ 5:24 pm
Location – UK
Firefox – displays 11PM, as reported by Prof Pullum.
Chromium – displays year numbers correctly.
set my location to Chicago
Firefox now displays correct year numbers.
Phoenix (which doesn;t observe DST) – correct year numbers.
Madrid (CET) – again correct year numbers.
Lisbon (GMT) – displays 11PM in Firefox.
So it only seems to affect countries in the GMT time zone.
Back to London, changed date to 28 March (after the European DST transition) – Firefox now displays 01AM.
Philip said,
March 16, 2016 @ 6:01 pm
Zizoz is obviously right.
J. W. Brewer said,
March 16, 2016 @ 6:36 pm
OK, I recall the general rhetorical contours of the H. Rap Brown quote but hadn't recalled that it deviated from the "apple" formulation. Do we think he was trying to make some subtle point for doing so, or was it just variation for variation's sake?
J. W. Brewer said,
March 16, 2016 @ 7:29 pm
Here's a crotchety letter to the editor of a supposedly highbrow publication chiding them for misquoting H. Rap Brown as saying "apple pie," with an equally crotchety author's response saying what's the big deal? http://www.nybooks.com/articles/1982/09/23/cherry-pie/
tangent said,
March 17, 2016 @ 1:27 am
This looks very similar to a Firefox issue three years ago:
https://github.com/mbostock/d3/issues/1196
Skip down towards the bottom for the meat of what was really going on there.
Yerushalmi said,
March 17, 2016 @ 1:43 am
"It's not Scots dialect: 1930 is known here as "1930"."
I am laughing so hard my wife is staring at me strangely.
Mark Mandel said,
March 17, 2016 @ 2:14 am
JW Brewer: Maybe because cherry pie filling is red, like blood?
Bob Ladd said,
March 17, 2016 @ 3:02 am
I'm not only in the same time zone as Geoff Pullum, but in the same town, and I get the expected dates on the x-axis, not 11 PM. Using Safari.
Hans Adler said,
March 17, 2016 @ 3:33 am
Clearly Deborah Pickett and tangent are on the right track. In my (German) timezone or in Chromium I can't see the bug, but I can reproduce it by setting my timezone to the UK one and using Firefox. I'm using a standard Linux variant, so all I have to do is enter "TZ = Europe/London" in a terminal window and then start Firefox (which must not be running at that point) from that terminal. Other methods apply on other operating systems.
The results are identical on Google's n-gram viewer and on http://jsfiddle.net/nrabinowitz/JTrnC/ – the JSFiddle page referenced by the old bug report that tangent found: incorrect display precisely in Firefox when using the problematic timezone setting.
The bug possibly appears only on certain days of certain years in certain timezones (perhaps just BST). For an hour or so I have been trying to isolate the problem, but I am having trouble resolving all the indirection in that JSFiddle code.
rosie said,
March 17, 2016 @ 3:54 am
England. (Still on GMT here.) No glitch; years display correctly. My search was
https://books.google.com/ngrams/graph?content=American+as+apple+pie%2CAmerican+as+cherry+pie&year_start=1920&year_end=2000&corpus=15&smoothing=3&share=&direct_url=t1%3B%2CAmerican%20as%20apple%20pie%3B%2Cc0%3B.t1%3B%2CAmerican%20as%20cherry%20pie%3B%2Cc0
Hans Adler said,
March 17, 2016 @ 5:17 am
The JavaScript code on the Google page is too hard for me to use for analysis. On the JSFiddle page I referred to above I found that it uses a certain date formatting function which for some reason is referred to as x.tickFormat(10) in the code. I put it into a variable for easier readability
f = x.tickFormat(10);
The function in principle – and also in practice on a system set to UTC – works as follows:
f(new Date("1912-01-01T00:00:00.000Z")) returns "1912"
f(new Date("1912-09-01T00:00:00.000Z")) returns "September"
f(new Date("1912-09-24T00:00:00.000Z")) returns "Sep 24"
f(new Date("1912-09-24T23:00:00.000Z")) returns "11 PM"
f(new Date("1912-09-24T11:30:00.000Z")) returns "11:30"
f(new Date("1912-09-24T11:30:00.000Z")) returns "11:30"
f(new Date("1912-09-24T11:30:05.000Z")) returns ":05"
f(new Date("1912-09-24T11:30:05.123Z")) returns ".123"
The problem now has to do with converting between UTC time and local time. E.g., in my temporary British timezone, what actually happens for the months is this:
f(new Date("1912-09-01T00:00:00.000Z")) returns "01 AM"
f(new Date("1912-08-31T23:00:00.000Z")) returns "September"
f(new Date("1912-02-01T00:00:00.000Z")) returns "February"
f(new Date("1912-01-31T23:00:00.000Z")) returns "11 PM"
It appears that somewhere on the way there is a date-related JavaScript function that behaves differently in Firefox than in Chrome and so causes the date objects to lose an hour relative to it. This makes every date object intended to represent a year, month or date appear as "11 pm", and makes every date object intended to represent an hour off by one hour. But all of this only applies to dates in the winter half.
Without analysing this further, we can't tell which browser is right. Possibly both are, since a lot of JavaScript date functionality is famously non-standardised and the library used here may well have a bug in its code that works around that problem. (My guess: The library code somewhere compares two hours using < without taking into account that hours wrap around at midnight – causing a bug that can only appear for timezones that consist of UTC with some daylight savings time regime. Basically that should be just BST/Western European Time and Morocco Time, which has different DST rules.)
Jen said,
March 17, 2016 @ 5:33 am
I'm in Scotland – Edinburgh University, even – and getting the years. Very odd.
Dave Orr said,
March 17, 2016 @ 12:20 pm
Thanks for the amazing debugging work to look into this, everyone. I'm a PM at Google working on NLP, and I'll see if I can get someone to look into this.
Vic said,
March 17, 2016 @ 2:47 pm
"It's not Scots dialect: 1930 is known here as "1930"."
I did a lot of work in Finland in the '70s (spent about 1,800 days in Helsinki over 10 years).
I've never been good at foreign languages – studied, but didn't really learn, Hebrew, Spanish, Latin, French, and German – but I tried to learn some Finnish anyway.
One day as I walked through a bookstore, I was able to understand one of the Finnish book titles, but by the time I realized this was special, I couldn't remember which one. So I went back to identify it.
"1984".
Rolyh said,
March 18, 2016 @ 4:39 am
Here in the Azores (GMT -1) it is displaying correctly on Firefox which would add strength to the idea that it is a GMT issue
Brett Reynolds said,
March 18, 2016 @ 5:11 am
This used to happen with Firefox for me. Resetting the browser seems to solve the problem.
January First-of-May said,
March 18, 2016 @ 7:05 am
I *think* I've seen a lot of problems like that in Excel tables at some point. Don't recall precisely.
I believe that, yes, the commenters mostly explained the reason correctly: a combination of a DST mismatch and a weird function that returns the last significant digit (so to say) of a date-time object as a short representation thereof.
I'm not sure why would the latter be needed, but I suppose it would probably say what is intended most of the time (unless your dates are much more precise than you need), and nobody really thought much about DST.
(Moscow City here. Our DST system went through lots of weird changes in the last few years; in all my travels of the last 20 years I've never been outside {UTC+2, UTC+3, UTC+4}, and over my lifetime – born 1992 – I've been in all three of these just in Moscow City.)
Sarah said,
March 20, 2016 @ 2:16 am
I'm in England, using Firefox, and it displays correctly for me.