Categories
Mozilla

New Asynchronous Storage API Has Landed

Some time in January I spec’ed out and API to allow the binding of multiple sets of parameters to a prepared statement and execute it asynchronously. This could be done in the past, but it was difficult and resource intensive to do (you would have to first clone the statement that you had and then bind and then pass all the clones to executeAsync). Say you want to insert a number of rows into a table now. Your code will look something like this:

// Note: db is the mozIStorageConnection for our database.
var stmt = db.createStatement("INSERT INTO table_name (table_value)" +
                              "VALUES (:value)");
var values = [
  "val1",
  "val2",
  "val3",
];
var array = stmt.newBindingParamsArray();
for (let i = 0; i < values.length; i++) {
  let params = array.newBindingParams();
  params.bindByName("value", values[i]);
  array.addParams(params);
}
stmt.bindParameters(array);
stmt.executeAsync(); // This will insert all the values into the table.

It's not the prettiest thing in the wold, and I hope to add some language helpers for JavaScript to make this even easier to use from JavaScript. Hope you find it useful!

Categories
Mozilla

Asynchronous Location Bar Searches

For those of you using the asynchronous location bar add-on, today’s nightly of both mozilla-central and mozilla-1.9.1 should show a reduction in memory use. Turns out there was a bit of a leak in mozStorage that would mean you’d get high memory usage that would never go down. My bad.

For those of your programmers out there, I have a word of advice. Always make sure you have a virtual destructor in your base class. If you don’t, you could spend days tracking down a leak that doesn’t seem to make sense. Of course, this probably would have been spotted earlier if we reported leaks in xpcshell unit tests.

Categories
Mozilla

Query Performance Monitoring

Over in bug 481261 I am adding support in mozStorage to warn when a query doesn’t use an index to sort. This is a debug only warning that uses NS_WARNING to tell you what query is at fault, and the total number of sort operations that were used on it.

Luckily, SQLite exposes a handy function that tells us this information. In general, you want to use a index in your ORDER BY clause so SQLite can use it to generate the order the results a given back. If you do not use an index, SQLite has to first get all the results in memory, then it sorts them, and then it starts to return results. If you expect a lot of results, this can get very expensive.

There were suggestions on the newsgroups to also add an API so debuggers such as ChromeBug or SQLite Manager could listen for these types of errors. I’ll be filing follow-up bugs for some of these suggestions later on.

Categories
Mozilla

More fsync and write Reduction

As you may or may not be aware, my personal mission as of late is to reduce the number of writes and fsyncs that Firefox makes, and move the ones that we do have to make off of the main thread. The primary target here has been Places, and the work is still continuing.

The Firefox team has been focusing on code sprints to get some small well scoped things done for Firefox 3.1 since we’ve got a bit more time. My latest sprint can be found over in bug 480211, where I’ve removed a write and fsync that we used to do after every page visited. If we had enough pages in history that were old enough, we would remove them from history. We now do this off of the main thread, asynchronously at the same time we flush data from our temporary tables to our permanent ones. The net result is the same number of writes and one less fsync. Additionally, the write is no longer done on the main thread.

Sadly, I couldn’t measure any real-world performance gains with my DTrace scripts – in fact I saw no change during several different runs of Tp3 with various places.sqlite files. It’s quite possible I did not have the conditions setup correctly to have pages expiring, and I could have spent a few more hours generating just the right places.sqlite file to demonstrate wins in the real world, but the theory behind the patch is pretty simple. The gain is pretty obvious.

Just another drop in the bucket of performance wins for Firefox. Stay tuned, as there is more to come!

Categories
Mozilla

Test Build: Asynchronous Location Bar Searches (Take 2)

It turns out that I neglected to make the all-important packages-static file changes that would actually result in a build that works for most folks in my previous test builds. I thought I had fixed that with the last build, but apparently I never submitted it to the try server (I had the line in my terminal all ready to just press enter though!).

Without further delay, here is the new test build.

I’ve also made a handy little add-on that lets you test this out. I haven’t tested this add-on extensively, but it should work out OK. If you think you see a bug, try the test build first before reporting it please.

Still want your feedback on if you think the results are faster, slower, or about the same, so please follow-up!