Categories
Mozilla

Startup Time in the Wild

Over the weekend I spent some serious time with my computer running a bunch of tests with standalone talos in 11 different situations. First, a disclaimer: these tests were only designed to give some insight on the areas we should focus on for the goal. Each of these tests was reproduced at least once before I moved onto the next one in order to make sure the numbers were stable.

The Tests

  • Clean profile. This is just the standard profile that we normally run Ts with on tinderbox. This is basically used a baseline for best possible performance.
  • Dirty profile. This is actually my daily profile, with eight tabs that will open through session restore during startup. Because of how talos works, these tabs don’t all have to load for the number to be generated. Even so, you’ll notice a substantial slowdown. Sadly, I fear I modified the profile I was using in a bad way because I can no longer reproduce the numbers I got (but the numbers recorded were reproduced four times before I moved on to the rest of the tests initially).
  • Bookmarks toolbar disabled. This is a variation on the dirty profile test that just disables the bookmarks toolbar.
  • No places. This is a variation on the dirty profile test that removes places files from the profile.
  • No sessionstore.js. This is a variation on the dirty profile test that removes sessionstore.js from the profile. This has the side effect of also not making the eight tabs load at startup.
  • No urlclassifier. This is a variation on the dirty profile test that removes the urlclassifier related files from the profile.
  • No cookies.sqlite. This is a variation on the dirty profile test that removes cookies.sqlite from the profile.
  • No extensions. This is a variation on the dirty profile test that removes all add-on manager bits in the profile.
  • No formhistory.sqlite. This is a variation on the dirty profile test that removes formhistory.sqlite from the profile.
  • No downloads.sqlite. This is a variation on the dirty profile test that removes downloads.sqlite from the profile.
  • No content-prefs.sqlite. This is a variation on the ditry profile test that removes content-prefs.sqlite from the profile.

Results

I’m going to let some graphs do the talking here. The first shows the raw test run data (which isn’t terribly interesting). The second compares the reported startup time for each test. You will probably want to click to zoom in.

Data of the startup time runs

Startup time of the various tests

Conclusions

It looks like the best wins that we can get are related to fixing session restore to not scale linearly with the number of tabs it is restoring, and reduce the startup time costs of loading places files and cookies.sqlite. It should be noted that this test was not measuring the load time for each tab, so something like BarTab would not help in this case. The other good news is that we already have work underway to make cookies.sqlite load time not hurt us so much during startup.

By Shawn Wilsher

The man behind the site.

14 replies on “Startup Time in the Wild”

Any advances are good, but I’m afraid I find it hard to get excited about gains of 200ms. The stuff that people I see most complaints about is in your “non-goals”.

If someone’s Firefox is taking 28.5 seconds to load up, they’re not really going to appreciate a reduction to 28.3 seconds…

I was expecting a massive difference between clean and ‘dirty’, but the difference is just 37%. It makes me somewhat wonder how ‘used’ your profile is. Extensions for example cost only 9ms; that doesn’t quite sound like you are running Adblock Plus. Don’t get me wrong because I very much like this initiative, but you might want to look for more ‘dirt’. As said in the first comment, 0.2 seconds isn’t exactly going to leave people in awe.
On a slow computer I used to have, Firefox (an earlier version) would become really sluggish, with the history toolbar becoming close to unusable, start up annoyingly slow and awesomebar very slow as well. I think it is those systems where real gains can be made, not on fast machines where speed differences between browsers are hardly noticeable.

Note that the “extensions” test specifically just removes all the files in the profile related to the add-ons manager. There were no add-ons enabled because I’m not looking for performance problems in add-ons. We have reports of long startup times from people who have no add-ons, and that is the target of this goal.

By Post Author

I understand about the extensions (I read the wiki page already). But I doubt those reports of long startup times are coming from people who have a startup time of 620ms.

Not that I have any better ideas of how to test, and the conclusions of looking at session restore and places seem reasonable. But if someone has a startup taking 20 seconds with no extensions, how can you conclude from your test that session restore is a similar chunk of their startup time, and that your 20% gain will translate to a 20% gain for them, rather than a 100ms gain (which would be <1% for them)?

I cannot possibly conclude that, nor did I say that that was my conclusion. We need to start somewhere, and this is certainly it. This week I’m going to be looking at some profiles that were given to us by users and see if a) I can reproduce their slow startup and b) I can get useful data from the profiles.

I also see people frequently stating that they have long startup time and then also brag about the tens or hundreds of tabs they have open on a day-to-day basis. The session restore performance hit scales linearly with the number of tabs it has open. I only had eight in this test.

And of course, I don’t expect there to be a magic bullet that will solve the startup problem. It is very likely a complicated problem without a simple solution.

By Post Author

Hi Shawn, my startup time has massively regressed between Firefox 3.6.8 and 4beta1 — some delayed startup event in Places is taking about 30 seconds and freezing the browser. It seems to happen on a timer after tabs open and windows raised, or is triggered if I focus the location bar.

It started misbehaving when I imported all my tagged delicious bookmarks so I could switch to using Firefox sync instead. DB files are large but not huge: Places.sqlite = 40 MB, formhistory.sqlite 1.5 MB. I can supply my places data (securely) if necessary.

I couldn’t find a bug for this for Beta 2 or in a search of Bugzilla; are you aware of any major Places startup regressions?

A few days ago, I looked in my profile-folder and found 34 files called sessionstore-1.js … sessionstore-34.js. They are probably just old sessionstore.js-files, but would these have any effect on startup time?

@AndersH – That won’t have any effect on startup time. We only ever read sessionstore.js. The others are probably there due to an addon. They might also be there due to timing of writes, which could cause your OS to add -N to the filename. They are all safe to delete (unless they are used by an extension, but if you don’t know why they’re there, it’s probably safe).

Unfortunately the goals, or more specifically the non-goals are probably the most accurate ‘metric’ for what users can expect from all this start up work: not very much!

How many users run a ‘clean profile’ ?
How many users run without any extensions?

If all the previous work on start up time – the Planet Mozilla evidence of which has vanished – was merely to improve clean start up and you are only getting to this dirty start up work now, with a margin of 20%, should users really expect a real-world faster start up from Firefox 4?

I appreciate that test conditions have to be consistent and scientific but really, to test start up performance without any add-ons is far from realistic surely?

When are we going to get a Firefox that actually does start up within 5 seconds in real-world conditions, Firefox 8?

Comments are closed.