Benchmarks

I haven't posted any browser benchmark since more than four years. For a good reason: if all contenders perform equally well, there's no need to benchmark them.

Of course, the recent excitement about Apple's Safari outpacing Chrome and Firefox still came to my attention. 😉 As it turned out, however, Safari managed to do that only in benchmarks developed by Apple, but not in those provided by Apple's competitors Google and Mozilla. This result seems to confirm qualified opinions as to which the available browser benchmarks are to be disregarded altogether.

Well, let's see what we have:

Google: Octane
Mozilla: Kraken

Now let's see what we've got:

System

Jetstream

Speedometer

Octane

Kraken (ms)

5: Office (i7 4790, Archlinux)

Chromium 44.0.2403.155

225.72±7.14

95.7±2.0

41571

749.8±0.6%

Firefox 40.0.2

215.21±8.08

61.0±2.1

35537

839.5±2.4%

4: Desktop (Xeon E3 1240 v2, Archlinux)

Chromium 44.0.2403.155

180.72±7.66

53.7±0.35

32280

919.6±1.7%

Firefox 40.0.2

164.03±1.44

29132

1030.3±4.1%

3: Notebook (Pentium P6200, Archlinux)

Chromium 44.0.2403.155

82.44±1.56

22.8±0.34

13167

2138.6±1.6%

Firefox 40.0.2

70.53±5.91

11944

2300.4±2.8%

2: Netbook (Atom N270, Debian Stretch)

Chromium 44.0.2403.107

17.28±1.37

5.15±0.07

2924

12985.7±3.7%

1: Tablet (ARM Cortex A9, Android 5.1.1)

Chrome 44.0.2403.133

15.46±0.92

7.64±0.19

2776

16248.3±5.1%

Chromium consistently performs better than Firefox across all benchmarks, even in Mozilla's own benchmark Kraken. The difference, however, is insignificant.

But that's not what I was actually interested in. What I really wanted to see was whether one can abuse these browser benchmarks for a kind of quick and dirty system benchmarking without the need to install anything. And as you see, all benchmarks scale fairly well:

../images/browser_benchmarks.svg

If we restrict ourselves to x86 for the moment, Jetstream (red) and Octane (blue) scale essentially identically across the entire range of systems. As a matter of fact, they even seem way too close if we suppose that Jetstream and octane are independent benchmarks. Kraken (yellow) scales very similarly, with the sole exception of the mini, for which it indicates only half of the performance than all other benchmarks. Finally, Speedometer (green) really seems to like the i4790. Perhaps it's making use of AVX2?

For the ARM architecture of the tablet, Jetstream and Octane are again almost identical, but Kraken suffers and Speedometer gains. No big deal, though: the notebook is still miles away. What's also interesting: a tablet from 2012 does not outperform a netbook from 2008, contrary to what the media want us to believe. But who believes them anyway anything anymore.

Compared to specialized benchmarks designed to test the number crunching performance of systems, the current ones reflect the average system performance we can expect in everyday situations. Particularly, of course, in browsing. 😉