[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Maposmatic-dev] Database update becoming critical

From: Maxime Petazzoni
Subject: Re: [Maposmatic-dev] Database update becoming critical
Date: Wed, 26 May 2010 09:13:52 +0200
User-agent: Mutt/1.5.20 (2009-06-14)

* Maxime Petazzoni <address@hidden> [2010-05-26 08:56:07]:

> But this is obviously not a perennial solution, and we should look to
> rapidly move to hourly our minute diffs using Osmosis. I'm currently
> investigation this solution in more detail. If one of you already had
> some information to share on Osmosis other than what's in task #10148
> <>, please do!

Apparently this process has been documented recently (April) on the OSM
wiki at

Using Osmosis, we can fetch the exact changes we need to apply to the
database from any timestamp to "now", and append these changes to the
database with osm2pgsql. After a simple two command setup:

  mkdir ~/.osmosis
  osmosis --read-replication-interval-init workingDirectory=~/.osmosis

The update is then as simple as creating the state file from the last
update timestamp, and piping osmosis into osm2pgsql to perform the

  wget \
    -O ~/.osmosis/state.txt
  osmosis --read-replication-interval workingDirectory=~/.osmosis   \
    --simplify-change --write-xml-change | osm2pgsql -a [our custom args]

With TIMESTAMP being a UTC timestamp in the form YYYY-MM-DDTHH:MM:SSZ. A
very simple hourly crontab could be created from this. We'd have to
check of course that this process has a significant time advantage. I
believe the more frequent we make the update process, the faster it
could run: I don't think the processing time grows linearly with the
update interval (aka it takes way longer than 60 times what it takes to
process a minute diff to process an hourly diff).

- Maxime
Maxime Petazzoni <>
 ``One by one, the penguins took away my sanity.''
Linux kernel and software developer at MontaVista Software

Attachment: signature.asc
Description: Digital signature

reply via email to

[Prev in Thread] Current Thread [Next in Thread]