Firefox 3 Memory Benchmarks and Comparison

You are wondering about the memory usage of web browsers. Many people load hundreds of web pages, often over periods of 3+ hours, and complain about the memory usage of Firefox, Safari, or Internet Explorer. Traditional benchmarks do not look at all the things you might do with a program, and we need real-world numbers over a period of hours. I developed a Windows Forms application in .NET called Memory Watcher that "watches" the system memory numbers. It uses a timer to poll the processes every three seconds. It then records every number, which allows us to keep track of each program's memory usage over time and with real-world usage. I collected over 1 million data points over 14+ hours.

Browser memory profiles

These results are from opening Memory Watcher and then using the browser between 9,000 and 11,000 seconds—close to 3 hours. Each browser is tested in a separate session, and there are brief periods of inactivity throughout the time period. The vertical axis is the memory used in MB, and the horizontal axis contains the memory "checkpoints" my program took (one every 3 seconds).

Benchmark details

The above profiles are not a direct comparison in any way, but they offer a visualization of trending in the memory behavior of the layout engines and interfaces. This is not a diagnosis or bug report. Let me show some important metrics of the above results.

The system is Windows Vista SP1, and the computer has 3.0+ GB of RAM. No plugins are disabled, but the Acrobat Reader and Java plugins were (presumably) not used. Flock is based on Firefox 2.0 but its memory usage is probably worse because it uses built-in extensions.

Just regular web pages. These aren't stress tests, and I probably never went over 4 windows in each browser, with at most 3 tabs in each window. I didn't look at many pages that are extremely heavy on images, and no "browser benchmark" style pages. Gmail was used on each browser.

Not just pages. It is hard for a regular benchmark to "simulate" a user actually clicking on things. Interactions with the user can greatly influence memory or performance. Having a responsive browser is probably more important than just having a "fast" one at showing pages.

Plugins included. My profiles include Flash and possibly other plugins. A browser might have memory issues with a plugin and that could cause a significant problem with the user experience. (Most Windows Vista crashes have been due to graphics cards, not Vista itself, for example.)

Real-life usage. An automation script will never give the same insight into performance over time as will this sort of profile. As developers, we want to make programs that work well for our users, and not just for tests. The tests capture the "rhythm" of software usage.

Final memory measurements

The data in this article are those reported by Windows Vista, but the exact individual numbers should not be compared to each other. Some browsers were tested slightly longer than others, and some different pages were loaded. That said, here are the final performance metrics.

About the "Memory Watcher"

Memory Watcher is a small program I wrote that records the memory usage of each process on the system every three seconds. It uses the PrivateMemorySize64 long value from the Process collection in .NET.

Simple. There are tools similar to this, offered on every platform, but they are not usually easy to use. Memory Watcher provides a super-easy way to monitor every process and silently work in the background.

Exports to spreadsheet. It exports the currently viewed data to a CSV file. These data are easily taken into Excel, and were used for the graphs in this article.

Implementation notes. The application uses a DataGridView control, and sets its DataSource property to a DataTable which is built from the object collection. It uses a Timer to poll the system every 3 seconds. It offers searching and filtering of processes using a TextBox.

Downloads. The program executable and the C# source code is available at my download site, which is linked to at the bottom of the page. You may need to click the Google Code link.

Updates—June 2009

Here we note that a second iteration of this article was published on June 20, 2009, featuring Google Chrome 3.0 developer channel release, Firefox 3.5 RC, Opera 10b, and Safari 4.0 final. The results in this new benchmark have similar results to this study, with Firefox's new version maintaining excellent memory efficiency.

(See Chrome and Firefox 3.5 Memory Usage.)

Conclusion

These profiles are meant to provide a picture of what the memory behavior of popular browsers is over a period of time, not to provide absolute benchmark times. Firefox 3.0 shows memory usage that is significantly lower than Firefox 2, which also does very well. Here is a summary of my results.

Safari 3.1: Safari on Windows shows extremely poor memory management, and I do not know whether it ever reaches a high water mark. If this is by design, it is certainly a design that looks inefficient and seems to contradict Apple's marketing.

Firefox 3.0: This browser exhibits memory usage that is by far lower than the others. It releases memory to the system and the trend line is nearly flat.

Flock, based on Firefox 2.0: Flock did very well and this browser and Firefox 2.0 could likely be run for long periods without causing many problems. The extensions probably reduced the efficiency somewhat.

Opera 9.5: Opera's performance was about as good as Firefox 2.0 (Flock), and it could likely be used for very lengthy sessions. However, Kestrel is certainly not a revolutionary or even notable technology in this arena.

Internet Explorer 8 Beta 1: IE did well in the profile, although a worrying trend in the data could indicate that it would keep escalating. However, this browser could likely sustain many hours of moderate usage.

Final thoughts. After browsing for 14 hours with these programs, and recording all the results into spreadsheets, the most memory efficient browser in my usage is very clear—Firefox 3.0 not only trumps its older version, but every other popular offering on Windows. This article may help other vendors rethink their marketing campaigns, and may prompt further improvements.

(Visit blog.pavlov.net.)

(Do not copy this page.)