.
Examples:
- gtfs-to-sql some-gtfs/*.txt | sponge | psql -b # import into PostgreSQL
- gtfs-to-sql -u -- some-gtfs/*.txt | gzip >gtfs.sql.gz # generate a gzipped SQL dump
+ import-gtfs-into-duckdb some-gtfs.duckdb some-gtfs/*.txt
[1] https://developers.google.com/transit/gtfs/reference/extended-route-types
[2] https://groups.google.com/g/gtfs-changes/c/keT5rTPS7Y0/m/71uMz2l6ke0J
```
-Some notable limitations mentioned in the [PostgreSQL 14 documentation on date/time types](https://www.postgresql.org/docs/14/datatype-datetime.html):
-
-> For `timestamp with time zone`, the internally stored value is always in UTC (Universal Coordinated Time, traditionally known as Greenwich Mean Time, GMT). An input value that has an explicit time zone specified is converted to UTC using the appropriate offset for that time zone.
-
-> When a `timestamp with time zone` value is output, it is always converted from UTC to the current `timezone` zone, and displayed as local time in that zone. To see the time in another time zone, either change `timezone` or use the `AT TIME ZONE` construct […].
-
-You can run queries with date+time values in any timezone (offset) and they will be processed correctly, but the output will always be in the database timezone (offset), unless you have explicitly used `AT TIME ZONE`.
+> [!TIP]
+> DuckDB will always store `timestamp with time zone` values as microsends since the [Unix epoch](https://en.wikipedia.org/wiki/Unix_time) (similar to UTC). An input value with an explicit offset specified (e.g. `2022-03-04T05:06:07+08:00`) is converted to the internal representation using the offset.
+> When the stored value is queried, it is always converted back into the current offset of the timezone specified by the `TimeZone` config. To see the time in another time zone, [change the `TimeZone` config](https://duckdb.org/docs/1.2/sql/data_types/timestamp#settings).
+> TLDR: You can run queries with date+time values in any timezone (offset) and they will be processed correctly.
### With Docker
*Note:* Just like the `npm`-installed variant, the Docker integration too assumes that your GTFS dataset consists of individual files (i.e. unzipped).
-Instead of installing via `npm`, you can use [the `ghcr.io/public-transport/gtfs-via-postgres` Docker image](https://github.com/public-transport/gtfs-via-postgres/pkgs/container/gtfs-via-postgres):
-
-```shell
-# variant A: use Docker image just to convert GTFS to SQL
-docker run --rm --volume /path/to/gtfs:/gtfs \
- ghcr.io/public-transport/gtfs-via-postgres --require-dependencies -- '/gtfs/*.csv' \
- | sponge | psql -b
-```
+Instead of installing via `npm`, you can use [the `ghcr.io/public-transport/gtfs-via-duckdb` Docker image](https://github.com/public-transport/gtfs-via-duckdb/pkgs/container/gtfs-via-duckdb):
*Note:* Remember to pass the `/gtfs/*.csv` glob as a string (with `'`), so that it gets evaluated *inside* the Docker container.
-With the code above, the `psql -b` process will run *outside* of the Docker container, so your host machine needs access to PostgreSQL.
-
-If you want to directly import the GTFS data *from within the Docker container*, you need add `psql` to the image and run it from inside. To do that, write a new Dockerfile that extends the `ghcr.io/public-transport/gtfs-via-postgres` image:
-
-```Dockerfile
-FROM ghcr.io/public-transport/gtfs-via-postgres
-ENV PGPORT=5432 PGUSER=postgres
-WORKDIR /gtfs
-# pass all arguments into gtfs-via-postgres, pipe output into psql:
-ENTRYPOINT ["/bin/sh", "-c", "gtfs-via-postgres $0 $@ | sponge | psql -b"]
-```
-
```shell
-# start PostgreSQL DB in another container "db"
-docker run --name db -p 5432:5432 -e POSTGRES_PASSWORD=password postgis/postgis
-
-# variant B: use Docker image to convert GTFS to SQL and import it directly
-docker build -t import-gtfs . # build helper Docker image from Dockerfile
docker run --rm --volume /path/to/gtfs:/gtfs \
- --link db -e PGHOST=db -e PGPASSWORD=password \
- import-gtfs --require-dependencies -- '/gtfs/*.csv'
+ ghcr.io/public-transport/gtfs-via-duckdb --require-dependencies -- '/gtfs/*.csv'
```
-### Importing a GTFS Schedule feed continuously
-
-[postgis-gtfs-importer](https://github.com/mobidata-bw/postgis-gtfs-importer) imports [GTFS Schedule](https://gtfs.org/schedule/) data into a [PostGIS](https://postgis.net) database using `gtfs-via-postgres`. It allows running a production service (e.g. an API) on top of programmatically re-imported data from a periodically changing GTFS feed without downtime.
-
-Because it works as [atomically](https://en.wikipedia.org/wiki/Atomicity_(database_systems)) as possible with PostgreSQL, it makes the import pipeline *robust*, even if an import fails or if simultaneous imports get started.
-
### Exporting data efficiently
-If you want to export data from the database, use the [`COPY` command](https://www.postgresql.org/docs/14/sql-copy.html); On an [M1 MacBook Air](https://en.wikipedia.org/wiki/MacBook_Air_(Apple_silicon)#Third_generation_(Retina_with_Apple_silicon)), PostgreSQL 14 can export about 500k `connections` rows per second.
+If you want to export data from the database, use the [`COPY` command](https://duckdb.org/docs/stable/sql/statements/copy).
```shell
-psql -c 'COPY (SELECT * FROM connections) TO STDOUT csv HEADER' >connections.csv
+duckdb -c 'COPY (SELECT * FROM connections) TO STDOUT csv HEADER' my-gtfs.duckdb >my-gtfs-connections.csv
```
-In the nested `SELECT` query, you can use features like `WHERE`, `ORDER BY` and `LIMIT`. Because `psql` passes on the exported data right away, you could stream it into another process.
-
### Querying stops by location efficiently
-If you want to find stops by (geo)location, run `gtfs-via-postgres` with `--stops-location-index`. This will create a [spatial index](https://postgis.net/workshops/postgis-intro/indexing.html) on `stops.stop_loc`, so that most [PostGIS functions & operators](https://postgis.net/docs/manual-3.2/reference.html#Measurement_Functions) make use of it.
-
-### GraphQL support
-
-The `--postgraphile` flag changes the SQL generated by `gtfs-via-postgres` slightly, so that you get a reasonably idiomatic GraphQL API out-of-the-box when running [PostGraphile](https://www.graphile.org/postgraphile/) v4 on it:
-
-```shell
-# import data into PostgreSQL with PostGraphile tweaks
-npm exec -- gtfs-to-sql -d --postgraphile -- gtfs/*.csv | sponge | psql -b
-```
-
-In line with the intended PostGraphile usage, `gtfs-via-postgres` will create a PostgreSQL role/user `postgraphile` with read-only access to the DB. You can set the `postgraphile`'s password with the `--postgraphile-password` option, or using the `$POSTGRAPHILE_PGPASSWORD` environment variable; By default, it will use (and log) a random password.
-
-`gtfs-via-postgres` *doesn't* specify PostGraphile as a regular dependency, but as `peerDependencies`, in order to stay lightweight for users who don't need the GraphQL interface. Some versions of some package managers install unmet peer dependencies, some don't. Let's make sure that PostGraphile (and its plugins) are installed:
-
-```shell
-npm install \
- postgraphile@^4.12 \
- @graphile-contrib/pg-simplify-inflector@^6.1 \
- @graphile/postgis@^0.2.0-0
-```
-
-The `serve-gtfs-via-graphql` helper script configures and runs PostGraphile. With `NODE_ENV=development`, it will
-
-- serve a fully configured [GraphiQL UI](https://graphql-dotnet.github.io/docs/getting-started/graphiql/) at `/graphiql`
-- provide more errors on database & query errors
-- allow [using PostgreSQL's `EXPLAIN` via GraphQL](https://www.graphile.org/postgraphile/debugging/#via-postgraphiql-explain)
-
-```
-# listens on port 3000, this can be changed using $PORT
-env NODE_ENV=development npm exec -- serve-gtfs-via-graphql
-```
-
-**As an example for the GraphQL API, check out the [test query](test/sample-gtfs-feed-postgraphile-test.graphql)** or open the [GraphiQL UI](https://github.com/graphql/graphiql) served at [`localhost:3000/graphiql`](http://localhost:3000/graphiql).
-
-### REST API support
-
-With the `--postgrest` flag, `gtfs-via-postgres` will augment the schema with a `web_anon` role and some comments, so that when running [PostgREST](https://postgrest.org/) on the database, you will get a powerful REST API.
-
-[read more](docs/postgrest.md)
+If you want to find stops by (geo)location, run `gtfs-via-duckdb` with `--stops-location-index`. This will create a [spatial index](https://duckdb.org/docs/stable/extensions/spatial/r-tree_indexes) on `stops.stop_loc`, so that most spatial queries can be done efficiently.
### more guides
-The [`docs` directory](docs) contains more instructions on how to use `gtfs-via-postgres`.
+The [`docs` directory](docs) contains more instructions on how to use `gtfs-via-duckdb`.
## Correctness vs. Speed regarding GTFS Time Values
@@ -338,24 +242,24 @@ This means that, in order to determine all *absolute* points in time where a par
Let's consider two examples:
-- A `departure_time` of `26:59:00` with a trip running on `2021-03-01`: The time, applied to this specific date, "extends" into the following day, so it actually departs at `2021-03-02T02:59+01`.
-- A departure time of `03:01:00` with a trip running on `2021-03-28`: This is when the standard -> DST switch happens in the `Europe/Berlin` timezone. Because the dep. time refers to noon - 12h (*not* to midnight), it actually happens at `2021-03-28T03:01+02` which is *not* `3h1m` after `2021-03-28T00:00+01`.
+- A `departure_time` of `26:59:00` with a trip running on `2021-03-01`: The time, applied to this specific date, "extends" into the following day, so it actually departs at `2021-03-02T02:59:00+01`.
+- A departure time of `03:01:00` with a trip running on `2021-03-28`: This is when the standard -> DST switch happens in the `Europe/Berlin` timezone. Because the dep. time refers to noon - 12h (*not* to midnight), it actually happens at `2021-03-28T03:01:00+02` which is *not* `3h1m` after `2021-03-28T00:00:00+01`.
-`gtfs-via-postgres` always prioritizes correctness over speed. Because it follows the GTFS semantics, when filtering `arrivals_departures` by *absolute* departure date+time, it cannot automatically filter `service_days` (which is `calendar` and `calendar_dates` combined), because **even a date *before* the date of the desired departure time frame might still end up *within*, when combined with a `departure_time` of e.g. `27:30:00`**; Instead, it has to consider all `service_days` and apply the `departure_time` to all of them to check if they're within the range.
+`gtfs-via-duckdb` always prioritizes correctness over speed. Because it follows the GTFS semantics, when filtering `arrivals_departures` by *absolute* departure date+time, it cannot automatically filter `service_days` (which is `calendar` and `calendar_dates` combined), because **even a date *before* the date of the desired departure time frame might still end up *within*, when combined with a `departure_time` of e.g. `27:30:00`**; Instead, it has to consider all `service_days` and apply the `departure_time` to all of them to check if they're within the range.
-However, if you determine your feed's largest `arrival_time`/`departure_time`, you can filter on `date` when querying `arrivals_departures`; This allows PostgreSQL to reduce the number of joins and calendar calculations by orders of magnitude, speeding up your queries significantly. `gtfs-via-postgres` provides two low-level helper functions `largest_arrival_time()` & `largest_departure_time()` for this, as well as two high-level helper functions `dates_filter_min(t_min)` & `dates_filter_max(t_max)` (see below).
+However, if you determine your feed's largest `arrival_time`/`departure_time`, you can filter on `date` when querying `arrivals_departures`; This allows DuckDB to reduce the number of joins and calendar calculations by orders of magnitude, speeding up your queries significantly. `gtfs-via-duckdb` provides a low-level helper table `largest_arr_dep_time` for this, as well as two high-level helper functions `dates_filter_min(t_min)` & `dates_filter_max(t_max)` (see below).
-For example, when querying all *absolute* departures at `de:11000:900120003` (*S Ostkreuz Bhf (Berlin)*) between `2022-03-23T12:30+01` and `2022-03-23T12:35+01` within the [2022-02-25 *VBB* feed](https://vbb-gtfs.jannisr.de/2022-02-25/), filtering by `date` speeds it up nicely (Apple M1, PostgreSQL 14.2):
+For example, when querying all *absolute* departures at `de:11000:900120003` (*S Ostkreuz Bhf (Berlin)*) between `2022-03-23T12:30:00+01` and `2022-03-23T12:35:00+01` within the [2025-05-21 *VBB* feed](https://vbb-gtfs.jannisr.de/2025-05-21/), filtering by `date` speeds it up nicely (Apple M2, DuckDB v1.3.0):
`station_id` filter | `date` filter | query time | nr of results
-|-|-|-
-`de:11000:900120003` | *none* | 230ms | ~574k
-`de:11000:900120003` | `2022-03-13` >= `date` < `2022-04-08` | 105ms | ~51k
-`de:11000:900120003` | `2022-03-23` >= `date` < `2022-03-24` | 55ms | ~2k
-`de:11000:900120003` | `2022-03-22` > `date` < `2022-03-24` | 55ms | ~2k
-*none* | *none* | 192s | 370m
-*none* | `2022-03-13` >= `date` < `2022-04-08` | 34s | ~35m
-*none* | `2022-03-22` > `date` < `2022-03-24` | 2.4s | ~1523k
+`de:11000:900120003` | *none* | todo | ~todok
+`de:11000:900120003` | `2022-03-13` >= `date` < `2022-04-08` | todo | ~todok
+`de:11000:900120003` | `2022-03-23` >= `date` < `2022-03-24` | todo | ~todok
+`de:11000:900120003` | `2022-03-22` > `date` < `2022-03-24` | todo | ~todok
+*none* | *none* | todo | todom
+*none* | `2022-03-13` >= `date` < `2022-04-08` | todo | ~todom
+*none* | `2022-03-22` > `date` < `2022-03-24` | todo | ~todok
Using `dates_filter_min(t_min)` & `dates_filter_max(t_max)`, we can easily filter by `date`. When filtering by `t_departure` (absolute departure date+time), `t_min` is the lower `t_departure` bound, whereas `t_max` is the upper bound. The VBB example above can be queried like this:
@@ -363,46 +267,45 @@ Using `dates_filter_min(t_min)` & `dates_filter_max(t_max)`, we can easily filte
SELECT *
FROM arrivals_departures
-- filter by absolute departure date+time
-WHERE t_departure >= '2022-03-23T12:30+01' AND t_departure <= '2022-03-23T12:35+01'
+WHERE t_departure >= '2022-03-23T12:30:00+01' AND t_departure <= '2022-03-23T12:35:00+01'
-- allow "cutoffs" by filtering by date
-AND "date" >= dates_filter_min('2022-03-23T12:30+01') -- evaluates to 2023-03-22
-AND "date" <= dates_filter_max('2022-03-23T12:35+01') -- evaluates to 2023-03-23
+AND "date" >= dates_filter_min('2022-03-23T12:30:00+01') -- evaluates to 2023-03-22
+AND "date" <= dates_filter_max('2022-03-23T12:35:00+01') -- evaluates to 2023-03-23
```
## Performance
-With all use cases I could think of, `gtfs-via-postgres` is reasonably fast. If there's a particular kind of query that you think should be faster, please [open an Issue](https://github.com/public-transport/gtfs-via-postgres/issues/new)!
+`gtfs-via-duckdb` is fast enough for most use cases I can think of. If there's a particular kind of query that you think should be faster, please [open an Issue](https://github.com/public-transport/gtfs-via-duckdb/issues/new)!
-The following benchmarks were run with the [2022-07-01 VBB GTFS dataset](https://vbb-gtfs.jannisr.de/2022-07-01/) (41k `stops`, 6m `stop_times`, 207m arrivals/departures) using `gtfs-via-postgres@4.7.4` and PostgreSQL 14.7 on an [M2](https://en.wikipedia.org/wiki/Apple_M2) laptop running macOS 12.6.8; All measurements are in milliseconds.
+The following benchmarks were run with the [2025-05-21 VBB GTFS dataset](https://vbb-gtfs.jannisr.de/2025-05-21/) (41k `stops`, 6m `stop_times`, 207m arrivals/departures) using `gtfs-via-duckdb@5.0.0` and DuckDB v1.3 on an [M2](https://en.wikipedia.org/wiki/Apple_M2) laptop running macOS 14.7.7; All measurements are in milliseconds.
| query | avg | min | p25 | p50 | p75 | p95 | p99 | max | iterations |
| - | - | - | - | - | - | - | - | - | - |
-| SELECT *
FROM stops
ORDER BY ST_Distance(stop_loc::geometry, ST_SetSRID(ST_MakePoint(9.7, 50.547), 4326)) ASC
LIMIT 100
| 15 | 14.982 | 15 | 15 | 15 | 15 | 15 | 15.488 | 100 |
-| SELECT *
FROM arrivals_departures
WHERE route_short_name = 'S1'
AND t_departure >= '2022-08-09T07:10+02' AND t_departure <= '2022-08-09T07:30+02'
AND date >= dates_filter_min('2022-08-09T07:10+02')
AND date <= dates_filter_max('2022-08-09T07:30+02')
| 61 | 60.901 | 61 | 61 | 61 | 61 | 62 | 61.778 | 100 |
-| SELECT *
FROM arrivals_departures
WHERE station_id = 'de:11000:900100001' -- S+U Friedrichstr. (Berlin)
AND t_departure >= '2022-08-09T07:10+02' AND t_departure <= '2022-08-09T07:30+02'
AND date >= dates_filter_min('2022-08-09T07:10+02')
AND date <= dates_filter_max('2022-08-09T07:30+02')
| 33 | 33.129 | 33 | 33 | 33 | 33 | 33 | 33.342 | 40 |
-| SELECT *
FROM arrivals_departures
WHERE station_id = 'de:11000:900100001' -- S+U Friedrichstr. (Berlin)
AND t_departure >= '2022-08-09T07:10+02' AND t_departure <= '2022-08-09T07:30+02'
AND date >= dates_filter_min('2022-08-09T07:10+02')
AND date <= dates_filter_max('2022-08-09T07:30+02')
AND stop_sequence = 0
| 5 | 4.548 | 5 | 5 | 5 | 5 | 5 | 4.598 | 50 |
-| SELECT *
FROM arrivals_departures
WHERE stop_id = 'de:11000:900100001::4' -- S+U Friedrichstr. (Berlin)
AND t_departure >= '2022-08-09T07:10+02' AND t_departure <= '2022-08-09T07:30+02'
AND date >= dates_filter_min('2022-08-09T07:10+02')
AND date <= dates_filter_max('2022-08-09T07:30+02')
| 8 | 8.038 | 8 | 8 | 8 | 8 | 8 | 8.164 | 100 |
-| SELECT *
FROM arrivals_departures
WHERE trip_id = '168977951'
AND date > '2022-08-08' AND date <= '2022-08-09'
| 2 | 1.878 | 2 | 2 | 2 | 2 | 2 | 1.911 | 100 |
-| SELECT count(*)
FROM arrivals_departures
WHERE stop_id = 'de:11000:900100001::4' -- S+U Friedrichstr. (Berlin)
| 58 | 57.485 | 58 | 58 | 58 | 58 | 58 | 57.789 | 100 |
-| SELECT count(*)
FROM arrivals_departures
WHERE stop_id = 'definitely-non-existent'
| 2 | 1.832 | 2 | 2 | 2 | 2 | 2 | 1.876 | 100 |
-| SELECT *
FROM arrivals_departures
WHERE t_departure >= '2022-08-09T07:10+02' AND t_departure <= '2022-08-09T07:30+02'
AND date >= dates_filter_min('2022-08-09T07:10+02'::timestamp with time zone)
AND date <= dates_filter_max('2022-08-09T07:30+02'::timestamp with time zone)
| 6310 | 6238.819 | 6241 | 6262 | 6311 | 6503 | 6560 | 6573.768 | 10 |
-| SELECT *
FROM arrivals_departures
WHERE t_departure >= '2022-08-09T07:10+02' AND t_departure <= '2022-08-09T07:30+02'
AND date >= '2022-08-08'
AND date <= '2022-08-09'
| 4931 | 4914.388 | 4925 | 4928 | 4937 | 4946 | 4948 | 4948.689 | 10 |
-| SELECT *
FROM connections
WHERE route_short_name = 'S1'
AND t_departure >= '2022-08-09T07:10+02' AND t_departure <= '2022-08-09T07:30+02'
AND date >= dates_filter_min('2022-08-09T07:10+02')
AND date <= dates_filter_max('2022-08-09T07:30+02')
| 164 | 163.018 | 163 | 164 | 164 | 164 | 165 | 166.568 | 100 |
-| SELECT *
FROM connections
WHERE from_station_id = 'de:11000:900100001' -- S+U Friedrichstr. (Berlin)
AND t_departure >= '2022-08-09T07:10+02' AND t_departure <= '2022-08-09T07:30+02'
AND date >= dates_filter_min('2022-08-09T07:10+02')
AND date <= dates_filter_max('2022-08-09T07:30+02')
| 59 | 58.137 | 58 | 58 | 59 | 60 | 61 | 61.461 | 40 |
-| SELECT *
FROM connections
WHERE from_station_id = 'de:11000:900100001' -- S+U Friedrichstr. (Berlin)
AND t_departure >= '2022-08-09T07:10+02' AND t_departure <= '2022-08-09T07:30+02'
AND date >= dates_filter_min('2022-08-09T07:10+02')
AND date <= dates_filter_max('2022-08-09T07:30+02')
AND from_stop_sequence = 0
| 7 | 7.439 | 7 | 7 | 7 | 7 | 7 | 7.49 | 50 |
-| SELECT *
FROM connections
WHERE from_stop_id = 'de:11000:900100001::4' -- S+U Friedrichstr. (Berlin)
AND t_departure >= '2022-08-09T07:10+02' AND t_departure <= '2022-08-09T07:30+02'
AND date >= dates_filter_min('2022-08-09T07:10+02')
AND date <= dates_filter_max('2022-08-09T07:30+02')
| 15 | 14.529 | 15 | 15 | 15 | 15 | 15 | 14.698 | 100 |
-| SELECT *
FROM connections
WHERE trip_id = '168977951'
AND date > '2022-08-08' AND date <= '2022-08-09'
| 3 | 2.86 | 3 | 3 | 3 | 3 | 3 | 2.931 | 100 |
-| SELECT count(*)
FROM connections
WHERE from_stop_id = 'de:11000:900100001::4' -- S+U Friedrichstr. (Berlin)
| 73 | 72.687 | 73 | 73 | 73 | 73 | 73 | 73.35 | 100 |
-| SELECT count(*)
FROM connections
WHERE from_stop_id = 'definitely-non-existent'
| 3 | 3.428 | 3 | 3 | 3 | 3 | 4 | 3.525 | 100 |
-| SELECT *
FROM connections
WHERE t_departure >= '2022-08-09T07:10+02' AND t_departure <= '2022-08-09T07:30+02'
AND date >= dates_filter_min('2022-08-09T07:10+02'::timestamp with time zone)
AND date <= dates_filter_max('2022-08-09T07:30+02'::timestamp with time zone)
ORDER BY t_departure
LIMIT 100
| 13127 | 13056.841 | 13086 | 13125 | 13170 | 13194 | 13199 | 13200.027 | 7 |
-| SELECT *
FROM connections
WHERE t_departure >= '2022-08-09T07:10+02' AND t_departure <= '2022-08-09T07:30+02'
AND date >= '2022-08-08'
AND date <= '2022-08-09'
ORDER BY t_departure
LIMIT 100
| 6417 | 6237.932 | 6346 | 6394 | 6512 | 6562 | 6570 | 6571.455 | 7 |
-| SELECT *
FROM stats_by_route_date
WHERE route_id = '17452_900' -- M4
AND date >= '2022-08-08' AND date <= '2022-08-14'
AND is_effective = true
| 2862 | 2853.972 | 2860 | 2863 | 2863 | 2867 | 2867 | 2866.798 | 10 |
-
+| SELECT *
FROM stops
ORDER BY ST_Distance(stop_loc::geometry, ST_Point(9.7, 50.547)) ASC
LIMIT 100
| 6.35 | 5.91 | 5.98 | 6.25 | 6.6 | 6.86 | 8.41 | 10.05 | 1576 |
+| SELECT *
FROM arrivals_departures
WHERE route_short_name = 'S1'
AND t_departure >= '2025-05-27T07:10:00+02' AND t_departure <= '2025-05-27T07:30:00+02'
AND date >= dates_filter_min('2025-05-27T07:10:00+02'::timestamp with time zone)
AND date <= dates_filter_max('2025-05-27T07:30+02'::timestamp with time zone)
| 305.15 | 260.52 | 303.8 | 307.73 | 312.2 | 320.64 | 326.84 | 328.44 | 33 |
+| SELECT *
FROM arrivals_departures
WHERE station_id = 'de:11000:900100001' -- S+U Friedrichstr. (Berlin)
AND t_departure >= '2025-05-27T07:10+02' AND t_departure <= '2025-05-27T07:30+02'
AND date >= dates_filter_min('2025-05-27T07:10+02')
AND date <= dates_filter_max('2025-05-27T07:30+02')
| 129.43 | 119.85 | 126.19 | 128.62 | 131.84 | 138.44 | 140.46 | 142 | 78 |
+| SELECT *
FROM arrivals_departures
WHERE station_id = 'de:11000:900100001' -- S+U Friedrichstr. (Berlin)
AND t_departure >= '2025-05-27T07:10+02' AND t_departure <= '2025-05-27T07:30+02'
AND date >= dates_filter_min('2025-05-27T07:10+02')
AND date <= dates_filter_max('2025-05-27T07:30+02')
AND stop_sequence = 0
| 81.42 | 65.73 | 79.48 | 82.11 | 84.33 | 87.26 | 89.64 | 102.97 | 123 |
+| SELECT *
FROM arrivals_departures
WHERE stop_id = 'de:11000:900100001::4' -- S+U Friedrichstr. (Berlin)
AND t_departure >= '2025-05-27T07:10+02' AND t_departure <= '2025-05-27T07:30+02'
AND date >= dates_filter_min('2025-05-27T07:10+02')
AND date <= dates_filter_max('2025-05-27T07:30+02')
| 83.79 | 64.57 | 82.15 | 84.64 | 85.83 | 91.36 | 95.79 | 97.08 | 120 |
+| SELECT *
FROM arrivals_departures
WHERE trip_id = '262623609' -- route_id=10144_109, route_short_name=S2
AND date = '2025-05-27'
| 14.25 | 12.38 | 13.42 | 13.98 | 14.84 | 16.12 | 18.98 | 21.77 | 702 |
+| SELECT count(*)
FROM arrivals_departures
WHERE stop_id = 'de:11000:900100001::4' -- S+U Friedrichstr. (Berlin)
| 70.9 | 67.54 | 69.09 | 70.1 | 72.47 | 75.73 | 77.24 | 78.83 | 142 |
+| SELECT count(*)
FROM arrivals_departures
WHERE stop_id = 'definitely-non-existent'
| 23.61 | 20.31 | 21.97 | 22.67 | 24.84 | 27.51 | 30.78 | 40.43 | 424 |
+| SELECT *
FROM arrivals_departures
WHERE t_departure >= '2025-05-27T07:10+02' AND t_departure <= '2025-05-27T07:30+02'
AND date >= '2025-05-25'
AND date <= '2025-05-27'
| 1269.86 | 1139.03 | 1254.52 | 1272.09 | 1318.94 | 1329.66 | 1331.44 | 1331.89 | 8 |
+| SELECT *
FROM arrivals_departures
WHERE t_departure >= '2025-05-27T07:10:00+02' AND t_departure <= '2025-05-27T07:30:00+02'
AND "date" >= dates_filter_min('2025-05-27T07:10:00+02'::timestamp with time zone)
AND "date" <= dates_filter_max('2025-05-27T07:30:00+02'::timestamp with time zone)
| 34148.21 | 32101.25 | 33459.12 | 34816.99 | 35171.69 | 35455.44 | 35512.2 | 35526.38 | 3 |
+| SELECT *
FROM connections
WHERE route_short_name = 'S1'
AND t_departure >= '2025-05-27T07:10+02' AND t_departure <= '2025-05-27T07:30+02'
AND date >= dates_filter_min('2025-05-27T07:10+02')
AND date <= dates_filter_max('2025-05-27T07:30+02')
| 8697.84 | 8629.78 | 8673.26 | 8716.73 | 8731.86 | 8743.96 | 8746.39 | 8746.99 | 3 |
+| SELECT *
FROM connections
WHERE from_station_id = 'de:11000:900194006' -- S Schöneweide/Sterndamm (Berlin)
AND t_departure >= '2025-05-27T07:10+02' AND t_departure <= '2025-05-27T07:30+02'
AND date >= dates_filter_min('2025-05-27T07:10+02')
AND date <= dates_filter_max('2025-05-27T07:30+02')
| 1154.01 | 1070.8 | 1115.77 | 1156.47 | 1168.38 | 1243.5 | 1281.37 | 1290.84 | 9 |
+| SELECT *
FROM connections
WHERE from_station_id = 'de:11000:900194006' -- S Schöneweide/Sterndamm (Berlin)
AND t_departure >= '2025-05-27T07:10+02' AND t_departure <= '2025-05-27T07:30+02'
AND date >= dates_filter_min('2025-05-27T07:10+02')
AND date <= dates_filter_max('2025-05-27T07:30+02')
AND from_stop_sequence_consec = 0
| 482.23 | 454.29 | 466.55 | 467.45 | 475.64 | 555.32 | 571.05 | 574.98 | 21 |
+| SELECT *
FROM connections
WHERE from_stop_id = 'de:11000:900100001::4' -- S+U Friedrichstr. (Berlin)
AND t_departure >= '2025-05-27T07:10+02' AND t_departure <= '2025-05-27T07:30+02'
AND date >= dates_filter_min('2025-05-27T07:10+02')
AND date <= dates_filter_max('2025-05-27T07:30+02')
| 885.14 | 835.29 | 869.24 | 875.76 | 909.79 | 922.32 | 923.64 | 923.97 | 12 |
+| SELECT *
FROM connections
WHERE trip_id = '262535123' -- route_id=17452_900 (M4)
AND date >= '2025-05-26' AND date <= '2025-06-01'
| 19.31 | 15.83 | 18.02 | 18.99 | 20.27 | 22.76 | 24.78 | 27.96 | 519 |
+| SELECT count(*)
FROM connections
WHERE from_stop_id = 'de:11000:900100001::4' -- S+U Friedrichstr. (Berlin)
| 341.42 | 263.96 | 340.65 | 346.83 | 350.72 | 355.91 | 358.76 | 359.65 | 30 |
+| SELECT count(*)
FROM connections
WHERE from_stop_id = 'definitely-non-existent'
| 343.5 | 314.1 | 319.13 | 345.04 | 354.63 | 362.52 | 463.4 | 503.94 | 30 |
+| SELECT *
FROM connections
WHERE t_departure >= '2025-05-27T07:10+02' AND t_departure <= '2025-05-27T07:30+02'
AND date >= dates_filter_min('2025-05-27T07:10+02'::timestamp with time zone)
AND date <= dates_filter_max('2025-05-27T07:30+02'::timestamp with time zone)
ORDER BY t_departure
LIMIT 100
| 1013055.35 | 986377.24 | 1026394.41 | 1009900.4 | 1026394.41 | 992028.36 | 1042228.66 | 1042888.42 | 3 |
+| SELECT *
FROM connections
WHERE t_departure >= '2025-05-27T07:10+02' AND t_departure <= '2025-05-27T07:30+02'
AND date >= '2025-05-25' AND date <= '2025-05-27'
ORDER BY t_departure
LIMIT 100
| 16347.21 | 16250.36 | 16285.17 | 16319.98 | 16395.63 | 16456.16 | 16468.27 | 16471.29 | 3 |
+| SELECT *
FROM stats_by_route_date
WHERE route_id = '17452_900' -- M4
AND date >= '2025-05-26' AND date <= '2025-06-01'
AND is_effective = true
| 4765.59 | 4704.49 | 4706.87 | 4709.25 | 4796.14 | 4865.64 | 4879.54 | 4883.02 | 3 |
## Related Projects
-There are some projects that are very similar to `gtfs-via-postgres`:
+There are some projects that are very similar to `gtfs-via-duckdb`:
### Node-GTFS
@@ -428,9 +331,9 @@ I don't use it because
There are several forks of the [original outdated project](https://github.com/cbick/gtfs_SQL_importer); [fitnr's fork](https://github.com/fitnr/gtfs-sql-importer) seems to be the most recent one.
-The project has a slightly different goal than `gtfs-via-postgres`: While `gtfs-sql-importer` is designed to import multiple versions of a GTFS dataset in an idempotent fashion, `gtfs-via-postgres` assumes that *one* (version of a) GTFS dataset is imported into *one* DB exactly once.
+The project has a slightly different goal than `gtfs-via-duckdb`: While `gtfs-sql-importer` is designed to import multiple versions of a GTFS dataset in an idempotent fashion, `gtfs-via-duckdb` assumes that *one* (version of a) GTFS dataset is imported into *one* DB exactly once.
-`gtfs-via-postgres` aims to provide more tools – e.g. the `arrivals_departures` & `connections` views – to help with the analysis of a GTFS dataset, whereas `gtfs-sql-importer` just imports the data.
+`gtfs-via-duckdb` aims to provide more tools – e.g. the `arrivals_departures` & `connections` views – to help with the analysis of a GTFS dataset, whereas `gtfs-sql-importer` just imports the data.
### other related projects
@@ -443,11 +346,12 @@ The project has a slightly different goal than `gtfs-via-postgres`: While `gtfs-
- [gtfs-lib](https://github.com/conveyal/gtfs-lib) – Java library & CLI for importing GTFS files into a PostgreSQL database.
- [gtfs-schema](https://github.com/tyleragreen/gtfs-schema) – PostgreSQL schemas for GTFS feeds. (plain SQL)
- [markusvalo/HSLtraffic](https://github.com/markusvalo/HSLtraffic) – Scripts to create a PostgreSQL database for HSL GTFS-data. (plain SQL)
+- [smohiudd/gtfs-parquet-duckdb-wasm](https://github.com/smohiudd/gtfs-parquet-duckdb-wasm) – Test visualization of GTFS data using DuckDB-Wasm ([blog post](http://saadiqm.com/gtfs-parquet-duckdb-wasm/))
## License
-This project is dual-licensed: **My ([@derhuerst](https://github.com/derhuerst)) contributions are licensed under the [*Prosperity Public License*](https://prosperitylicense.com), [contributions of other people](https://github.com/public-transport/gtfs-via-postgres/graphs/contributors) are licensed as [Apache 2.0](https://apache.org/licenses/LICENSE-2.0)**.
+This project is dual-licensed: **My ([@derhuerst](https://github.com/derhuerst)) contributions are licensed under the [*Prosperity Public License*](https://prosperitylicense.com), [contributions of other people](https://github.com/public-transport/gtfs-via-duckdb/graphs/contributors) are licensed as [Apache 2.0](https://apache.org/licenses/LICENSE-2.0)**.
> This license allows you to use and share this software for noncommercial purposes for free and to try this software for commercial purposes for thirty days.
@@ -458,6 +362,6 @@ This project is dual-licensed: **My ([@derhuerst](https://github.com/derhuerst))
## Contributing
-If you have a question or need support using `gtfs-via-postgres`, please double-check your code and setup first. If you think you have found a bug or want to propose a feature, use [the issues page](https://github.com/public-transport/gtfs-via-postgres/issues).
+If you have a question or need support using `gtfs-via-duckdb`, please double-check your code and setup first. If you think you have found a bug or want to propose a feature, use [the issues page](https://github.com/public-transport/gtfs-via-duckdb/issues).
By contributing, you agree to release your modifications under the [Apache 2.0 license](LICENSE-APACHE).
diff --git a/scripts/run-postgraphile.js b/scripts/run-postgraphile.js
deleted file mode 100755
index bf74ee8..0000000
--- a/scripts/run-postgraphile.js
+++ /dev/null
@@ -1,84 +0,0 @@
-#!/usr/bin/env node
-
-const {createServer} = require('http')
-const {postgraphile} = require('postgraphile')
-const postgisPlugin = require('@graphile/postgis').default
-const simplifyInflectorPlugin = require('@graphile-contrib/pg-simplify-inflector')
-
-const DEV = process.env.NODE_ENV === 'development'
-const PROD = !DEV
-const PORT = process.env.PORT ? parseInt(process.env.PORT) : 3000
-const SCHEMA = process.env.PGSCHEMA || 'public'
-
-const pg = postgraphile({}, SCHEMA, {
- appendPlugins: [
- // PostGIS support for PostGraphile
- postgisPlugin,
-
- // Simplifies the graphile-build-pg inflector to trim the `ByFooIdAndBarId` from relations
- simplifyInflectorPlugin,
- ],
- graphileBuildOptions: {
- pgSimplifyAllRows: false,
- pgShortPk: false,
- },
-
- pgSettings: async () => ({
- // With `timestamptz` (a.k.a. `timestamp with time zone`), PostgreSQL *doesn't* store the timezone (offset) specified on input; Instead, it always converts to UTC.
- // When querying a `timestamptz` value, it converts to the local timezone (offset) of the client's session or database server.
- // Because we loose the timezone offset information *anyways*, we configure PostGraphile to give predictable results by letting PostgreSQL always convert to UTC.
- timezone: 'UTC',
- }),
-
- // [Experimental] Determines if the 'Explain' feature in GraphiQL can be used to show the user the SQL statements that were executed. Set to a boolean to enable all users to use this, or to a function that filters each request to determine if the request may use Explain. DO NOT USE IN PRODUCTION unless you're comfortable with the security repurcussions of doing so.
- allowExplain: DEV,
-
- // Enables classic ids for Relay support. Instead of using the field name nodeId for globally unique ids, PostGraphile will instead use the field name id for its globally unique ids. This means that table id columns will also get renamed to rowId.
- classicIds: true,
-
- // Turns off GraphQL query logging. By default PostGraphile will log every GraphQL query it processes along with some other information. Set this to true (recommended in production) to disable that feature.
- disableQueryLog: PROD,
-
- // By default, JSON and JSONB fields are presented as strings (JSON encoded) from the GraphQL schema. Setting this to true (recommended) enables raw JSON input and output, saving the need to parse / stringify JSON manually.
- dynamicJson: true,
-
- // Set this to true to add some enhancements to GraphiQL; intended for development usage only (automatically enables with subscriptions and live).
- enhanceGraphiql: DEV,
-
- // Set this to true to enable the GraphiQL interface.
- graphiql: true,
-
- // Extends the error response with additional details from the Postgres error. Can be any combination of ['hint', 'detail', 'errcode']. Default is [].
- extendedErrors: DEV ? ['hint', 'detail', 'errcode'] : [],
-
- // Set false to exclude filters, orderBy, and relations that would be expensive to access due to missing indexes. Changing this from true to false is a breaking change, but false to true is not. The default is true.
- ignoreIndexes: false,
-
- // Set false (recommended) to exclude fields, queries and mutations that are not available to any possible user (determined from the user in connection string and any role they can become); set this option true to skip these checks and create GraphQL fields and types for everything. The default is true, in v5 the default will change to false.
- ignoreRBAC: false,
-
- // Some one-to-one relations were previously detected as one-to-many - should we export 'only' the old relation shapes, both new and old but mark the old ones as 'deprecated' (default), or 'omit' (recommended) the old relation shapes entirely.
- legacyRelations: 'omit',
-
- // If none of your RETURNS SETOF compound_type functions mix NULLs with the results then you may set this false to reduce the nullables in the GraphQL schema.
- setofFunctionsContainNulls: false,
-
- // Enables adding a stack field to the error response. Can be either the boolean true (which results in a single stack string) or the string json (which causes the stack to become an array with elements for each line of the stack). Recommended in development, not recommended in production.
- showErrorStack: DEV,
-
- // Should we use relay pagination, or simple collections?
- // "omit" (default)
- // relay connections only, "only" (not recommended)
- // simple collections only (no Relay connections), "both" - both.
- simpleCollections: 'omit',
-})
-
-const server = createServer(pg)
-server.listen(PORT, (err) => {
- if (err) {
- console.error(err)
- process.exit(1)
- }
- const {port} = server.address()
- console.info(`PostGraphile listening on port ${port}`)
-})
diff --git a/test/amtrak-gtfs-2021-10-06.sh b/test/amtrak-gtfs-2021-10-06.sh
index 0adcfc7..83a3f7f 100755
--- a/test/amtrak-gtfs-2021-10-06.sh
+++ b/test/amtrak-gtfs-2021-10-06.sh
@@ -11,21 +11,19 @@ env | grep '^PG' || true
unzip -q -j -n amtrak-gtfs-2021-10-06.zip -d amtrak-gtfs-2021-10-06
ls -lh amtrak-gtfs-2021-10-06
-psql -c 'create database amtrak_2021_10_06'
-export PGDATABASE='amtrak_2021_10_06'
+path_to_db="$(mktemp -d -t gtfs)/amtrak-gtfs-2021-10-06.duckdb"
-../cli.js -d --trips-without-shape-id --schema amtrak \
+../cli.js -d --trips-without-shape-id \
--import-metadata \
--stats-by-route-date=view \
--stats-by-agency-route-stop-hour=view \
--stats-active-trips-by-hour=view \
- --postgrest \
- -- amtrak-gtfs-2021-10-06/*.txt \
- | sponge | psql -b
+ "$path_to_db" \
+ -- amtrak-gtfs-2021-10-06/*.txt
query=$(cat << EOF
select extract(epoch from t_arrival)::integer as t_arrival
-from amtrak.arrivals_departures
+from arrivals_departures
where stop_id = 'BHM' -- Birmingham
and date = '2021-11-26'
order by t_arrival
@@ -33,26 +31,20 @@ EOF
)
# 2021-11-26T15:15:00-05:00
-arr1=$(psql --csv -t -c "$query" | head -n 1)
+arr1=$(duckdb -csv -noheader -c "$query" "$path_to_db" | head -n 1)
if [[ "$arr1" != "1637957700" ]]; then
echo "invalid 1st t_arrival: $arr1" 1>&2
exit 1
fi
# 2021-11-27T13:45:00-05:00
-arrN=$(psql --csv -t -c "$query" | tail -n 1)
+arrN=$(duckdb -csv -noheader -c "$query" "$path_to_db" | tail -n 1)
if [[ "$arrN" != "1638038700" ]]; then
echo "invalid 2nd t_arrival: $arrN" 1>&2
exit 1
fi
-version=$(psql --csv -t -c "SELECT split_part(amtrak.gtfs_via_postgres_version(), '.', 1)" | tail -n 1)
-if [[ "$version" != "4" ]]; then
- echo "invalid gtfs_via_postgres_version(): $version" 1>&2
- exit 1
-fi
-
-fMin=$(psql --csv -t -c "SELECT amtrak.dates_filter_min('2021-11-27T13:45:00-06')" | tail -n 1)
+fMin=$(duckdb -csv -noheader -c "SELECT dates_filter_min('2021-11-27T13:45:00-06')" "$path_to_db" | tail -n 1)
if [[ "$fMin" != "2021-11-24" ]]; then
echo "invalid dates_filter_min(…): $fMin" 1>&2
exit 1
@@ -60,13 +52,13 @@ fi
acelaStatQuery=$(cat << EOF
SELECT nr_of_trips, nr_of_arrs_deps
-FROM amtrak.stats_by_route_date
+FROM stats_by_route_date
WHERE route_id = '40751' -- Acela
AND date = '2021-11-26'
AND is_effective = True
EOF
)
-acelaStat=$(psql --csv -t -c "$acelaStatQuery" | tail -n 1)
+acelaStat=$(duckdb -csv -noheader -c "$acelaStatQuery" "$path_to_db" | tail -n 1)
if [[ "$acelaStat" != "16,190" ]]; then
echo "invalid stats for route 40751 (Acela) on 2021-11-26: $acelaStat" 1>&2
exit 1
@@ -74,13 +66,13 @@ fi
acelaPhillyStatQuery=$(cat << EOF
SELECT nr_of_arrs
-FROM amtrak.stats_by_agency_route_stop_hour
+FROM stats_by_agency_route_stop_hour
WHERE route_id = '40751' -- Acela
AND stop_id = 'PHL' -- Philadelphia
-AND effective_hour = '2022-07-24T09:00-05'
+AND effective_hour = '2022-07-24 09:00:00-05:00'
EOF
)
-acelaPhillyStat=$(psql --csv -t -c "$acelaPhillyStatQuery" | tail -n 1)
+acelaPhillyStat=$(duckdb -csv -noheader -c "$acelaPhillyStatQuery" "$path_to_db" | tail -n 1)
if [[ "$acelaPhillyStat" != "2" ]]; then
echo "invalid stats for route 40751 (Acela) at PHL (Philadelphia) on 2021-11-26: $acelaPhillyStat" 1>&2
exit 1
@@ -88,54 +80,20 @@ fi
nrOfActiveTripsQuery=$(cat << EOF
SELECT nr_of_active_trips
-FROM amtrak.stats_active_trips_by_hour
-WHERE "hour" = '2021-11-26T04:00-05'
+FROM stats_active_trips_by_hour
+WHERE "hour" = '2021-11-26 04:00:00-05:00'
EOF
)
# Note: I'm not sure if 127 is correct, but it is in the right ballpark. 🙈
# The following query yields 150 connections, and it doesn't contain those who depart earlier and arrive later.
# SELECT DISTINCT ON (trip_id) *
# FROM amtrak.connections
-# WHERE t_departure >= '2021-11-26T02:00-05'
-# AND t_arrival <= '2021-11-26T06:00-05'
-nrOfActiveTrips=$(psql --csv -t -c "$nrOfActiveTripsQuery" | tail -n 1)
+# WHERE t_departure >= '2021-11-26 02:00:00-05:00'
+# AND t_arrival <= '2021-11-26 06:00:00-05:00'
+nrOfActiveTrips=$(duckdb -csv -noheader -c "$nrOfActiveTripsQuery" "$path_to_db" | tail -n 1)
if [[ "$nrOfActiveTrips" != "127" ]]; then
echo "unexpected no. of active trips at 2021-11-26T04:00-05: $nrOfActiveTrips" 1>&2
exit 1
fi
-# kill child processes on exit
-# https://stackoverflow.com/questions/360201/how-do-i-kill-background-processes-jobs-when-my-shell-script-exits/2173421#2173421
-trap 'exit_code=$?; kill -- $(jobs -p); exit $exit_code' SIGINT SIGTERM EXIT
-
-env \
- PGRST_DB_SCHEMAS=amtrak \
- PGRST_DB_ANON_ROLE=web_anon \
- PGRST_ADMIN_SERVER_PORT=3001 \
- PGRST_LOG_LEVEL=info \
- postgrest &
- # docker run --rm -i \
- # -p 3000:3000 -p 3001:3001 \
- # -e PGHOST=host.docker.internal -e PGUSER -e PGPASSWORD -e PGDATABASE \
- # postgrest/postgrest &
-sleep 3
-
-health_status="$(curl 'http://localhost:3001/live' -I -fsS | grep -o -m1 -E '[0-9]{3}')"
-if [ "$health_status" != '200' ]; then
- 1>&2 echo "/live: expected 200, got $health_status"
- exit 1
-fi
-
-stops_url='http://localhost:3000/stops?stop_name=ilike.%25palm%25&limit=1&order=stop_id.asc'
-stops_status="$(curl "$stops_url" -H 'Accept: application/json' -I -fsS | grep -o -m1 -E '[0-9]{3}')"
-if [ "$stops_status" != '200' ]; then
- 1>&2 echo "$stops_url: expected 200, got $stops_status"
- exit 1
-fi
-stop_id="$(curl "$stops_url" -H 'Accept: application/json' -fsS | jq -rc '.[0].stop_id')"
-if [ "$stop_id" != 'PDC' ]; then
- 1>&2 echo "$stops_url: expected PDC, got $stop_id"
- exit 1
-fi
-
echo 'works ✔'
diff --git a/test/calendar-dates-only.sh b/test/calendar-dates-only.sh
index 1014ad1..e11dc4f 100755
--- a/test/calendar-dates-only.sh
+++ b/test/calendar-dates-only.sh
@@ -8,12 +8,11 @@ set -x
env | grep '^PG' || true
-psql -c 'create database calendar_dates_only'
-export PGDATABASE='calendar_dates_only'
+path_to_db="$(mktemp -d -t gtfs)/calendar-dates-only.duckdb"
../cli.js -d --trips-without-shape-id -- \
- calendar-dates-only/*.txt \
- | sponge | psql -b
+ "$path_to_db" \
+ calendar-dates-only/*.txt
query=$(cat << EOF
select extract(epoch from t_arrival)::integer as t_arrival
@@ -24,14 +23,14 @@ EOF
)
# 2019-07-15T15:30:00+02:00
-arr1=$(psql --csv -t -c "$query" | head -n 1)
+arr1=$(duckdb -csv -noheader -c "$query" "$path_to_db" | head -n 1)
if [[ "$arr1" != "1563197400" ]]; then
echo "invalid 1st t_arrival: $arr1" 1>&2
exit 1
fi
# 2019-07-20T15:30:00+02:00
-arrN=$(psql --csv -t -c "$query" | tail -n 1)
+arrN=$(duckdb -csv -noheader -c "$query" "$path_to_db" | tail -n 1)
if [[ "$arrN" != "1563629400" ]]; then
echo "invalid 2nd t_arrival: $arrN" 1>&2
exit 1
@@ -43,7 +42,7 @@ from arrivals_departures
where agency_id IS NULL
EOF
)
-agency_id_null_count="$(psql --csv -t -c "$agency_id_null")"
+agency_id_null_count="$(duckdb -csv -noheader -c "$agency_id_null" "$path_to_db")"
if [[ "$agency_id_null_count" != "0" ]]; then
echo ">0 rows with agency_id = null" 1>&2
exit 1
@@ -57,7 +56,7 @@ FROM arrivals_departures
ORDER BY stop_id, trip_id
EOF
)
-wheelchair_boarding_rows="$(psql --csv -t -c "$wheelchair_boarding_query")"
+wheelchair_boarding_rows="$(duckdb -csv -noheader -c "$wheelchair_boarding_query" "$path_to_db")"
wheelchair_boarding_expected="$(echo -e "airport,accessible\nairport-1,not_accessible\nlake,no_info_or_inherit\nmuseum,no_info_or_inherit")"
if [[ "$wheelchair_boarding_rows" != "$wheelchair_boarding_expected" ]]; then
echo "invalid wheelchair_boarding values" 1>&2
diff --git a/test/index.sh b/test/index.sh
index a40fa08..d8512f0 100755
--- a/test/index.sh
+++ b/test/index.sh
@@ -6,15 +6,14 @@ set -o pipefail
cd "$(dirname $0)"
set -x
-psql -t -c 'SELECT version()'
+duckdb --version
./calendar-dates-only.sh
./sample-gtfs-feed.sh
./amtrak-gtfs-2021-10-06.sh
-./postgraphile.sh
./routes-without-agency-id.sh
./stops-without-level-id.sh
./invalid-empty-agency-id.sh
-./multiple-schemas.sh
+./multiple-datasets.sh
echo -e "\n\n✔︎ tests passing"
diff --git a/test/invalid-empty-agency-id.sh b/test/invalid-empty-agency-id.sh
index c4dbba8..9245389 100755
--- a/test/invalid-empty-agency-id.sh
+++ b/test/invalid-empty-agency-id.sh
@@ -9,8 +9,9 @@ set -x
# Refer to https://github.com/public-transport/gtfs-via-postgres/issues/45 for context.
# The "core" bug: A feed without routes.agency_id should be importable.
-if ../cli.js -d --trips-without-shape-id -s -- \
- invalid-empty-agency-id/*.txt >/dev/null; then
+# However, this only applies if there is exactly one route. If there are >1 routes, every route must have an agency_id.
+if ../cli.js -d --trips-without-shape-id -s -- ':memory:' \
+ invalid-empty-agency-id/*.txt; then
echo "import didn't fail" 1>&2
exit 1
else
@@ -20,6 +21,6 @@ fi
# A related bug: With --routes-without-agency-id, lib/deps.js *does not* specify routes to depend on agency.
# *In some cases*, this causes agency to be processed *after* routes, causing the routes processing to fail.
# see also https://github.com/public-transport/gtfs-via-postgres/issues/45#issuecomment-1632649826
-../cli.js -d --routes-without-agency-id --trips-without-shape-id -s -- \
- invalid-empty-agency-id/*.txt >/dev/null
+../cli.js -d --routes-without-agency-id --trips-without-shape-id -s -- ':memory:' \
+ invalid-empty-agency-id/*.txt
echo 'did not fail even with --routes-without-agency-id ✔'
diff --git a/test/multiple-datasets.sh b/test/multiple-datasets.sh
new file mode 100755
index 0000000..fd17846
--- /dev/null
+++ b/test/multiple-datasets.sh
@@ -0,0 +1,138 @@
+#!/bin/bash
+
+set -e
+set -u
+set -o pipefail
+cd "$(dirname $0)"
+set -x
+
+env | grep '^PG' || true
+
+unzip -q -j -n amtrak-gtfs-2021-10-06.zip -d amtrak-gtfs-2021-10-06
+ls -lh amtrak-gtfs-2021-10-06
+
+db_dir="$(mktemp -d -t gtfs)"
+path_to_db1="$db_dir/multiple-schemas-1.duckdb"
+path_to_db2="$db_dir/multiple-schemas-2.duckdb"
+
+shopt -s extglob
+
+../cli.js -d --trips-without-shape-id \
+ "$path_to_db1" \
+ -- amtrak-gtfs-2021-10-06/!(transfers).txt
+
+../cli.js -d --trips-without-shape-id \
+ "$path_to_db2" \
+ -- amtrak-gtfs-2021-10-06/*.txt
+
+shopt -u extglob
+
+query_prefix=$(cat << EOF
+ATTACH DATABASE '$path_to_db1' AS one (READ_ONLY);
+ATTACH DATABASE '$path_to_db2' AS two (READ_ONLY);
+SET search_path = 'one,two';
+EOF
+)
+
+tables_query=$(cat << EOF
+$query_prefix
+SELECT
+ (table_catalog || '.' || table_name) AS table_name
+FROM information_schema.tables
+WHERE table_schema = 'main'
+AND table_catalog = ANY(['one', 'two'])
+ORDER BY table_catalog, table_name;
+EOF
+)
+tables_rows=$(duckdb -csv -noheader -c "$tables_query")
+# note that one.transfers is missing
+tables_expected=$(cat << EOF
+one.agency
+one.arrivals_departures
+one.calendar
+one.calendar_dates
+one.connections
+one.feed_info
+one.frequencies
+one.largest_arr_dep_time
+one.routes
+one.service_days
+one.stop_times
+one.stops
+one.trips
+one.valid_lang_codes
+one.valid_timezones
+two.agency
+two.arrivals_departures
+two.calendar
+two.calendar_dates
+two.connections
+two.feed_info
+two.frequencies
+two.largest_arr_dep_time
+two.routes
+two.service_days
+two.stop_times
+two.stops
+two.transfers
+two.trips
+two.valid_lang_codes
+two.valid_timezones
+EOF
+)
+if [[ "$tables_rows" != "$tables_expected" ]]; then
+ echo "unexpected list of tables" 1>&2
+ exit 1
+fi
+
+# https://dba.stackexchange.com/a/72656
+nr_of_unequal_stops=$(cat << EOF
+$query_prefix
+SELECT count(*)
+FROM one.stops a
+FULL OUTER JOIN two.stops b ON (
+ a.stop_id = b.stop_id
+)
+WHERE (
+ a.stop_code IS DISTINCT FROM b.stop_code
+ OR a.stop_name IS DISTINCT FROM b.stop_name
+ OR a.stop_desc IS DISTINCT FROM b.stop_desc
+ OR a.stop_loc IS DISTINCT FROM b.stop_loc
+ OR a.zone_id IS DISTINCT FROM b.zone_id
+ OR a.stop_url IS DISTINCT FROM b.stop_url
+ OR a.location_type::TEXT IS DISTINCT FROM b.location_type::TEXT
+ OR a.parent_station IS DISTINCT FROM b.parent_station
+ OR a.stop_timezone IS DISTINCT FROM b.stop_timezone
+ OR a.wheelchair_boarding::TEXT IS DISTINCT FROM b.wheelchair_boarding::TEXT
+ OR a.level_id IS DISTINCT FROM b.level_id
+ OR a.platform_code IS DISTINCT FROM b.platform_code
+)
+EOF
+)
+
+unequal_stops_1=$(duckdb -csv -noheader -c "$nr_of_unequal_stops" | head -n 1)
+if [[ "$unequal_stops_1" -ne 0 ]]; then
+ 1>&2 echo "$unequal_stops_1 unequal stops between one.stops & two.stops"
+ exit 1
+fi
+
+# # put an incompatible version
+# duckdb -c "$(cat << EOF
+# CREATE OR REPLACE FUNCTION public.gtfs_via_duckdb_import_version()
+# RETURNS TEXT
+# AS \$\$
+# SELECT '0.1.2'
+# \$\$
+# LANGUAGE SQL
+# EOF
+# )"
+
+# # expect another import to fail
+# if ../cli.js -d --trips-without-shape-id \
+# "$path_to_db" \
+# -- amtrak-gtfs-2021-10-06/*.txt; then
+# 1>&2 echo "re-import with incompatible version didn't fail"
+# exit 1
+# fi
+
+echo 'works ✔'
diff --git a/test/multiple-schemas.sh b/test/multiple-schemas.sh
deleted file mode 100755
index 7ddabbd..0000000
--- a/test/multiple-schemas.sh
+++ /dev/null
@@ -1,79 +0,0 @@
-#!/bin/bash
-
-set -e
-set -u
-set -o pipefail
-cd "$(dirname $0)"
-set -x
-
-env | grep '^PG' || true
-
-unzip -q -j -n amtrak-gtfs-2021-10-06.zip -d amtrak-gtfs-2021-10-06
-ls -lh amtrak-gtfs-2021-10-06
-
-psql -c 'create database multiple_schemas'
-export PGDATABASE='multiple_schemas'
-
-../cli.js -d --trips-without-shape-id \
- --schema one \
- -- amtrak-gtfs-2021-10-06/*.txt \
- | sponge | psql -b
-
-../cli.js -d --trips-without-shape-id \
- --schema two \
- -- amtrak-gtfs-2021-10-06/*.txt \
- | sponge | psql -b
-
-# https://dba.stackexchange.com/a/72656
-nr_of_unequal_stops=$(cat << EOF
-SELECT count(*)
-FROM one.stops a
-FULL OUTER JOIN two.stops b ON (
- a.stop_id = b.stop_id
-)
-WHERE (
- a.stop_code IS DISTINCT FROM b.stop_code
- OR a.stop_name IS DISTINCT FROM b.stop_name
- OR a.stop_desc IS DISTINCT FROM b.stop_desc
- OR a.stop_loc IS DISTINCT FROM b.stop_loc
- OR a.zone_id IS DISTINCT FROM b.zone_id
- OR a.stop_url IS DISTINCT FROM b.stop_url
- OR a.location_type::TEXT IS DISTINCT FROM b.location_type::TEXT
- OR a.parent_station IS DISTINCT FROM b.parent_station
- OR a.stop_timezone IS DISTINCT FROM b.stop_timezone
- OR a.wheelchair_boarding::TEXT IS DISTINCT FROM b.wheelchair_boarding::TEXT
- OR a.level_id IS DISTINCT FROM b.level_id
- OR a.platform_code IS DISTINCT FROM b.platform_code
-)
-EOF
-)
-
-unequal_stops_1=$(psql --csv -t -c "$nr_of_unequal_stops" | head -n 1)
-if [[ "$unequal_stops_1" -ne 0 ]]; then
- 1>&2 echo "$unequal_stops_1 unequal stops between one.stops & two.stops"
- exit 1
-fi
-
-# todo: assert that more tables are equal?
-
-# put an incompatible version
-psql -c "$(cat << EOF
-CREATE OR REPLACE FUNCTION public.gtfs_via_postgres_import_version()
-RETURNS TEXT
-AS \$\$
- SELECT '0.1.2'
-\$\$
-LANGUAGE SQL
-EOF
-)"
-
-# expect another import to fail
-if ../cli.js -d --trips-without-shape-id \
- --schema three \
- -- amtrak-gtfs-2021-10-06/*.txt \
- | sponge | psql -b; then
- 1>&2 echo "re-import with incompatible version didn't fail"
- exit 1
-fi
-
-echo 'works ✔'
diff --git a/test/postgraphile.sh b/test/postgraphile.sh
deleted file mode 100755
index f06bce1..0000000
--- a/test/postgraphile.sh
+++ /dev/null
@@ -1,31 +0,0 @@
-#!/bin/bash
-
-set -e
-set -u
-set -o pipefail
-cd "$(dirname $0)"
-set -x
-
-env | grep '^PG' || true
-
-psql -c 'create database postgraphile'
-export PGDATABASE='postgraphile'
-
-../cli.js -d --trips-without-shape-id --postgraphile -- \
- ../node_modules/sample-gtfs-feed/gtfs/*.txt \
- | sponge | psql -b
-
-# kill child processes on exit
-# https://stackoverflow.com/questions/360201/how-do-i-kill-background-processes-jobs-when-my-shell-script-exits/2173421#2173421
-trap 'exit_code=$?; kill -- $(jobs -p); exit $exit_code' SIGINT SIGTERM EXIT
-
-../scripts/run-postgraphile.js &
-sleep 2
-
-body=$(node -e 'process.stdout.write(JSON.stringify({query: fs.readFileSync("sample-gtfs-feed-postgraphile-test.graphql", {encoding: "utf8"})}))')
-actual_path="$(mktemp -t sample-gtfs-feed-postgraphile-test-XXX)"
-curl -X POST 'http://localhost:3000/graphql' -H 'Content-Type: application/json' -H 'Accept: application/json' --data "$body" -fsS | jq -r --tab . >"$actual_path"
-
-git diff --exit-code sample-gtfs-feed-postgraphile-test.res.json "$actual_path"
-
-echo 'works ✔'
diff --git a/test/routes-without-agency-id.sh b/test/routes-without-agency-id.sh
index 51e1530..c63e296 100755
--- a/test/routes-without-agency-id.sh
+++ b/test/routes-without-agency-id.sh
@@ -7,7 +7,7 @@ cd "$(dirname $0)"
set -x
../cli.js -d --routes-without-agency-id -- \
- ../node_modules/sample-gtfs-feed/gtfs/*.txt \
- >/dev/null
+ ':memory:' \
+ ../node_modules/sample-gtfs-feed/gtfs/*.txt
echo 'works ✔'
diff --git a/test/sample-gtfs-feed-postgraphile-test.graphql b/test/sample-gtfs-feed-postgraphile-test.graphql
deleted file mode 100644
index bccbf0d..0000000
--- a/test/sample-gtfs-feed-postgraphile-test.graphql
+++ /dev/null
@@ -1,115 +0,0 @@
-query SampleGtfsFeedPostgraphileTest {
- stopByStopId(stopId: "airport-1") {
- translatedStopName(language: "de-DE")
- stopId
- stopLoc {
- geojson
- latitude
- longitude
- }
- }
-
- routeByRouteId(routeId: "B") {
- agency {
- agencyName
- agencyEmail
- }
- routeShortName
- routeLongName
- at: translatedRouteLongName(language: "de-AT")
- de: translatedRouteLongName(language: "de-DE")
- trips(first: 3, orderBy: TRIP_ID_ASC) {
- nodes {
- tripId
- }
- }
- connections(orderBy: PRIMARY_KEY_ASC, offset: 3, first: 3) {
- nodes {
- tripId
- fromStopSequence
- fromStopId
- tDeparture
- tArrival
- toStopId
- toStopSequence
- }
- }
- }
- bOutboundOnWeekends: tripByTripId(tripId: "b-outbound-on-weekends") {
- translatedTripHeadsign(language: "de-DE")
- }
- aDowntownAllDay: tripByTripId(tripId: "a-downtown-all-day") {
- shape {
- shape {
- __typename
- geojson
- }
- }
- }
-
- # stop_times-based -> no frequencies_{row,it}
- aOutboundAllDay20190301ArrDep: arrivalDepartureByArrivalDepartureId(
- id: "YS1vdXRib3VuZC1hbGwtZGF5:MjAxOS0wMy0wMQ==:Mw==:LTE=:LTE="
- ) {
- # tripId: "a-outbound-all-day"
- # date: "2019-03-01"
- # stopSequence: 3
- # frequenciesRow: -1
- # frequenciesIt: -1
- arrivalDepartureId
- tripId
- date
- stopSequence
- frequenciesRow
- frequenciesIt
- }
- # frequencies-based -> has frequencies_{row,it}
- bDowntownOnWorkingDays20190608ArrDep: arrivalDepartureByArrivalDepartureId(
- id: "Yi1kb3dudG93bi1vbi13b3JraW5nLWRheXM=:MjAxOS0wMy0wOA==:Mw==:MQ==:Mg=="
- ) {
- # tripId: "b-downtown-on-working-days"
- # date: "2019-06-08"
- # stopSequence: 3
- # frequenciesRow: 1
- # frequenciesIt: 2
- arrivalDepartureId
- tripId
- date
- stopSequence
- frequenciesRow
- frequenciesIt
- }
-
- # stop_times-based -> no frequencies_{row,it}
- aOutboundAllDay20190301Con: connectionByConnectionId(
- id: "YS1vdXRib3VuZC1hbGwtZGF5:MjAxOS0wMy0wMQ==:Mw==:LTE=:LTE="
- ) {
- # tripId: "a-outbound-all-day"
- # date: "2019-03-01"
- # fromStopSequence: 3
- # frequenciesRow: -1
- # frequenciesIt: -1
- connectionId
- tripId
- date
- fromStopSequence
- frequenciesRow
- frequenciesIt
- }
- # frequencies-based -> has frequencies_{row,it}
- bDowntownOnWorkingDays20190608Con: connectionByConnectionId(
- id: "Yi1kb3dudG93bi1vbi13b3JraW5nLWRheXM=:MjAxOS0wMy0wOA==:Mw==:MQ==:Mg=="
- ) {
- # tripId: "b-downtown-on-working-days"
- # date: "2019-06-08"
- # fromStopSequence: 3
- # frequenciesRow: 1
- # frequenciesIt: 2
- connectionId
- tripId
- date
- fromStopSequence
- frequenciesRow
- frequenciesIt
- }
-}
diff --git a/test/sample-gtfs-feed-postgraphile-test.res.json b/test/sample-gtfs-feed-postgraphile-test.res.json
deleted file mode 100644
index 76359a1..0000000
--- a/test/sample-gtfs-feed-postgraphile-test.res.json
+++ /dev/null
@@ -1,2316 +0,0 @@
-{
- "data": {
- "stopByStopId": {
- "translatedStopName": "Gleis 1",
- "stopId": "airport-1",
- "stopLoc": {
- "geojson": {
- "type": "Point",
- "coordinates": [
- 13.5087,
- 52.36396
- ]
- },
- "latitude": 52.36396,
- "longitude": 13.5087
- }
- },
- "routeByRouteId": {
- "agency": {
- "agencyName": "Full Transit Agency",
- "agencyEmail": "contact@fta.example.org"
- },
- "routeShortName": "Babbage",
- "routeLongName": "Charles Babbage Tram Line",
- "at": "Tram-Linie Charles Babbage",
- "de": null,
- "trips": {
- "nodes": [
- {
- "tripId": "b-downtown-on-weekends"
- },
- {
- "tripId": "b-downtown-on-working-days"
- },
- {
- "tripId": "b-outbound-on-weekends"
- }
- ]
- },
- "connections": {
- "nodes": [
- {
- "tripId": "b-downtown-on-weekends",
- "fromStopSequence": 3,
- "fromStopId": "lake",
- "tDeparture": "2019-03-09T12:24:00+00:00",
- "tArrival": "2019-03-09T12:30:00+00:00",
- "toStopId": "center",
- "toStopSequence": 5
- },
- {
- "tripId": "b-downtown-on-weekends",
- "fromStopSequence": 1,
- "fromStopId": "airport",
- "tDeparture": "2019-03-10T12:14:00+00:00",
- "tArrival": "2019-03-10T12:22:00+00:00",
- "toStopId": "lake",
- "toStopSequence": 3
- },
- {
- "tripId": "b-downtown-on-weekends",
- "fromStopSequence": 3,
- "fromStopId": "lake",
- "tDeparture": "2019-03-10T12:24:00+00:00",
- "tArrival": "2019-03-10T12:30:00+00:00",
- "toStopId": "center",
- "toStopSequence": 5
- }
- ]
- }
- },
- "bOutboundOnWeekends": {
- "translatedTripHeadsign": "Babbage (auswärts)"
- },
- "aDowntownAllDay": {
- "shape": {
- "shape": {
- "__typename": "GeometryLineString",
- "geojson": {
- "type": "LineString",
- "coordinates": [
- [
- 13.510294914,
- 52.364833832
- ],
- [
- 13.510567665,
- 52.364398956
- ],
- [
- 13.510860443,
- 52.363952637
- ],
- [
- 13.511548042,
- 52.362854004
- ],
- [
- 13.511612892,
- 52.362743378
- ],
- [
- 13.511850357,
- 52.362812042
- ],
- [
- 13.513009071,
- 52.363082886
- ],
- [
- 13.513717651,
- 52.363246918
- ],
- [
- 13.514398575,
- 52.363361359
- ],
- [
- 13.516216278,
- 52.363788605
- ],
- [
- 13.516494751,
- 52.363868713
- ],
- [
- 13.516823769,
- 52.364009857
- ],
- [
- 13.516993523,
- 52.364112854
- ],
- [
- 13.517116547,
- 52.364208221
- ],
- [
- 13.517197609,
- 52.364322662
- ],
- [
- 13.517261505,
- 52.364448547
- ],
- [
- 13.517277718,
- 52.364532471
- ],
- [
- 13.517285347,
- 52.364704132
- ],
- [
- 13.517237663,
- 52.365009308
- ],
- [
- 13.517251968,
- 52.365158081
- ],
- [
- 13.517328262,
- 52.365364075
- ],
- [
- 13.517384529,
- 52.365451813
- ],
- [
- 13.517477036,
- 52.365539551
- ],
- [
- 13.517616272,
- 52.365650177
- ],
- [
- 13.517773628,
- 52.365726471
- ],
- [
- 13.518079758,
- 52.365856171
- ],
- [
- 13.518387794,
- 52.365940094
- ],
- [
- 13.528774261,
- 52.368408203
- ],
- [
- 13.529670715,
- 52.368545532
- ],
- [
- 13.530094147,
- 52.368579865
- ],
- [
- 13.5308218,
- 52.368587494
- ],
- [
- 13.531106949,
- 52.368598938
- ],
- [
- 13.531417847,
- 52.368621826
- ],
- [
- 13.531955719,
- 52.36869812
- ],
- [
- 13.532168388,
- 52.368759155
- ],
- [
- 13.532450676,
- 52.368862152
- ],
- [
- 13.53266716,
- 52.368961334
- ],
- [
- 13.532931328,
- 52.369121552
- ],
- [
- 13.533116341,
- 52.369255066
- ],
- [
- 13.533249855,
- 52.36938858
- ],
- [
- 13.533371925,
- 52.369533539
- ],
- [
- 13.533464432,
- 52.369682312
- ],
- [
- 13.533542633,
- 52.369838715
- ],
- [
- 13.533593178,
- 52.370014191
- ],
- [
- 13.533617973,
- 52.370185852
- ],
- [
- 13.533589363,
- 52.370334625
- ],
- [
- 13.533475876,
- 52.370624542
- ],
- [
- 13.533353806,
- 52.370826721
- ],
- [
- 13.533203125,
- 52.371002197
- ],
- [
- 13.532802582,
- 52.371387482
- ],
- [
- 13.532670021,
- 52.37153244
- ],
- [
- 13.532507896,
- 52.371768951
- ],
- [
- 13.532444,
- 52.371963501
- ],
- [
- 13.5324049,
- 52.372131348
- ],
- [
- 13.53239727,
- 52.37229538
- ],
- [
- 13.532422066,
- 52.372528076
- ],
- [
- 13.532460213,
- 52.372646332
- ],
- [
- 13.532538414,
- 52.372817993
- ],
- [
- 13.532709122,
- 52.373149872
- ],
- [
- 13.534140587,
- 52.375667572
- ],
- [
- 13.534313202,
- 52.375961304
- ],
- [
- 13.534439087,
- 52.376140594
- ],
- [
- 13.534526825,
- 52.376251221
- ],
- [
- 13.534785271,
- 52.376514435
- ],
- [
- 13.535042763,
- 52.376712799
- ],
- [
- 13.535244942,
- 52.376853943
- ],
- [
- 13.535474777,
- 52.376983643
- ],
- [
- 13.535713196,
- 52.377109528
- ],
- [
- 13.536309242,
- 52.377346039
- ],
- [
- 13.53663826,
- 52.377441406
- ],
- [
- 13.537053108,
- 52.377536774
- ],
- [
- 13.537810326,
- 52.377681732
- ],
- [
- 13.53807354,
- 52.377750397
- ],
- [
- 13.538312912,
- 52.377830505
- ],
- [
- 13.538555145,
- 52.377925873
- ],
- [
- 13.538812637,
- 52.378055573
- ],
- [
- 13.538974762,
- 52.37815094
- ],
- [
- 13.5391922,
- 52.378314972
- ],
- [
- 13.539357185,
- 52.378479004
- ],
- [
- 13.539421082,
- 52.378543854
- ],
- [
- 13.539493561,
- 52.378623962
- ],
- [
- 13.539569855,
- 52.378723145
- ],
- [
- 13.539703369,
- 52.379005432
- ],
- [
- 13.539748192,
- 52.379161835
- ],
- [
- 13.539772034,
- 52.37940979
- ],
- [
- 13.539751053,
- 52.379619598
- ],
- [
- 13.539697647,
- 52.379798889
- ],
- [
- 13.539621353,
- 52.379974365
- ],
- [
- 13.539505959,
- 52.380153656
- ],
- [
- 13.539352417,
- 52.380329132
- ],
- [
- 13.539167404,
- 52.380493164
- ],
- [
- 13.538882256,
- 52.380710602
- ],
- [
- 13.536517143,
- 52.382324219
- ],
- [
- 13.536241531,
- 52.382499695
- ],
- [
- 13.535950661,
- 52.382644653
- ],
- [
- 13.535591125,
- 52.3828125
- ],
- [
- 13.535319328,
- 52.382923126
- ],
- [
- 13.535028458,
- 52.383018494
- ],
- [
- 13.534606934,
- 52.383136749
- ],
- [
- 13.53421402,
- 52.383220673
- ],
- [
- 13.533993721,
- 52.38325882
- ],
- [
- 13.533719063,
- 52.383296967
- ],
- [
- 13.533379555,
- 52.383335114
- ],
- [
- 13.53301239,
- 52.383358002
- ],
- [
- 13.532653809,
- 52.383365631
- ],
- [
- 13.53222084,
- 52.383361816
- ],
- [
- 13.531785011,
- 52.383354187
- ],
- [
- 13.531435013,
- 52.383361816
- ],
- [
- 13.531114578,
- 52.383388519
- ],
- [
- 13.530774117,
- 52.383441925
- ],
- [
- 13.530474663,
- 52.383522034
- ],
- [
- 13.530198097,
- 52.383605957
- ],
- [
- 13.529940605,
- 52.383716583
- ],
- [
- 13.529669762,
- 52.383857727
- ],
- [
- 13.529401779,
- 52.384044647
- ],
- [
- 13.529109955,
- 52.38432312
- ],
- [
- 13.52870369,
- 52.384784698
- ],
- [
- 13.528428078,
- 52.38508606
- ],
- [
- 13.528366089,
- 52.385158539
- ],
- [
- 13.524540901,
- 52.389453888
- ],
- [
- 13.524550438,
- 52.389503479
- ],
- [
- 13.524573326,
- 52.389541626
- ],
- [
- 13.524604797,
- 52.389583588
- ],
- [
- 13.524658203,
- 52.389625549
- ],
- [
- 13.525242805,
- 52.389953613
- ],
- [
- 13.525495529,
- 52.390113831
- ],
- [
- 13.525518417,
- 52.390159607
- ],
- [
- 13.525501251,
- 52.390201569
- ],
- [
- 13.525468826,
- 52.390254974
- ],
- [
- 13.525419235,
- 52.390304565
- ],
- [
- 13.524431229,
- 52.391159058
- ],
- [
- 13.523122787,
- 52.392383575
- ],
- [
- 13.522995949,
- 52.392505646
- ],
- [
- 13.522948265,
- 52.392536163
- ],
- [
- 13.52287674,
- 52.392559052
- ],
- [
- 13.522799492,
- 52.392566681
- ],
- [
- 13.522711754,
- 52.392566681
- ],
- [
- 13.521859169,
- 52.392444611
- ],
- [
- 13.521745682,
- 52.392436981
- ],
- [
- 13.521669388,
- 52.392440796
- ],
- [
- 13.521622658,
- 52.392456055
- ],
- [
- 13.521595955,
- 52.392478943
- ],
- [
- 13.521548271,
- 52.392578125
- ],
- [
- 13.522637367,
- 52.392738342
- ],
- [
- 13.522878647,
- 52.392772675
- ],
- [
- 13.523015022,
- 52.392837524
- ],
- [
- 13.523111343,
- 52.392879486
- ],
- [
- 13.523198128,
- 52.392917633
- ],
- [
- 13.523303032,
- 52.392993927
- ],
- [
- 13.523317337,
- 52.393058777
- ],
- [
- 13.523306847,
- 52.393119812
- ],
- [
- 13.523284912,
- 52.393169403
- ],
- [
- 13.522663116,
- 52.393768311
- ],
- [
- 13.521858215,
- 52.394523621
- ],
- [
- 13.521655083,
- 52.394649506
- ],
- [
- 13.521375656,
- 52.39491272
- ],
- [
- 13.520638466,
- 52.395599365
- ],
- [
- 13.520013809,
- 52.396232605
- ],
- [
- 13.519786835,
- 52.396499634
- ],
- [
- 13.51952076,
- 52.396839142
- ],
- [
- 13.519312859,
- 52.397209167
- ],
- [
- 13.519210815,
- 52.397247314
- ],
- [
- 13.519133568,
- 52.397315979
- ],
- [
- 13.519043922,
- 52.397338867
- ],
- [
- 13.518992424,
- 52.397354126
- ],
- [
- 13.518731117,
- 52.397266388
- ],
- [
- 13.518521309,
- 52.397186279
- ],
- [
- 13.518030167,
- 52.396968842
- ],
- [
- 13.517698288,
- 52.397338867
- ],
- [
- 13.51756382,
- 52.397563934
- ],
- [
- 13.517389297,
- 52.397800446
- ],
- [
- 13.516566277,
- 52.398674011
- ],
- [
- 13.515673637,
- 52.399570465
- ],
- [
- 13.514561653,
- 52.400661469
- ],
- [
- 13.514300346,
- 52.400932312
- ],
- [
- 13.513332367,
- 52.401851654
- ],
- [
- 13.51246357,
- 52.40272522
- ],
- [
- 13.510783195,
- 52.40435791
- ],
- [
- 13.510543823,
- 52.404605865
- ],
- [
- 13.510230064,
- 52.404914856
- ],
- [
- 13.50899601,
- 52.406147003
- ],
- [
- 13.508612633,
- 52.406547546
- ],
- [
- 13.50774765,
- 52.407478333
- ],
- [
- 13.506917953,
- 52.408348083
- ],
- [
- 13.505527496,
- 52.409721375
- ],
- [
- 13.505458832,
- 52.40978241
- ],
- [
- 13.505138397,
- 52.41009903
- ],
- [
- 13.503731728,
- 52.411491394
- ],
- [
- 13.503533363,
- 52.411678314
- ],
- [
- 13.502279282,
- 52.412883759
- ],
- [
- 13.501524925,
- 52.413482666
- ],
- [
- 13.501321793,
- 52.413619995
- ],
- [
- 13.500832558,
- 52.413936615
- ],
- [
- 13.50038147,
- 52.4141922
- ],
- [
- 13.49997139,
- 52.414409637
- ],
- [
- 13.499858856,
- 52.414455414
- ],
- [
- 13.499188423,
- 52.414749146
- ],
- [
- 13.498696327,
- 52.41493988
- ],
- [
- 13.497921944,
- 52.415218353
- ],
- [
- 13.497368813,
- 52.415431976
- ],
- [
- 13.496650696,
- 52.415706635
- ],
- [
- 13.496446609,
- 52.415782928
- ],
- [
- 13.496009827,
- 52.415969849
- ],
- [
- 13.495700836,
- 52.416107178
- ],
- [
- 13.495515823,
- 52.416194916
- ],
- [
- 13.495312691,
- 52.416297913
- ],
- [
- 13.494745255,
- 52.41658783
- ],
- [
- 13.49464035,
- 52.416652679
- ],
- [
- 13.494258881,
- 52.416881561
- ],
- [
- 13.493819237,
- 52.417167664
- ],
- [
- 13.493548393,
- 52.417369843
- ],
- [
- 13.493290901,
- 52.417572021
- ],
- [
- 13.493026733,
- 52.417778015
- ],
- [
- 13.492693901,
- 52.418022156
- ],
- [
- 13.492493629,
- 52.418174744
- ],
- [
- 13.492147446,
- 52.418441772
- ],
- [
- 13.490100861,
- 52.420017242
- ],
- [
- 13.489993095,
- 52.420093536
- ],
- [
- 13.489733696,
- 52.420288086
- ],
- [
- 13.489574432,
- 52.420440674
- ],
- [
- 13.48927021,
- 52.420764923
- ],
- [
- 13.489129066,
- 52.420928955
- ],
- [
- 13.488491058,
- 52.421710968
- ],
- [
- 13.488237381,
- 52.421993256
- ],
- [
- 13.487900734,
- 52.422344208
- ],
- [
- 13.487172127,
- 52.422939301
- ],
- [
- 13.486559868,
- 52.423408508
- ],
- [
- 13.486092567,
- 52.423770905
- ],
- [
- 13.48562336,
- 52.424152374
- ],
- [
- 13.485471725,
- 52.424255371
- ],
- [
- 13.485077858,
- 52.424537659
- ],
- [
- 13.484401703,
- 52.425022125
- ],
- [
- 13.483383179,
- 52.425769806
- ],
- [
- 13.483257294,
- 52.425865173
- ],
- [
- 13.482924461,
- 52.426101685
- ],
- [
- 13.482698441,
- 52.426265717
- ],
- [
- 13.480806351,
- 52.427612305
- ],
- [
- 13.479895592,
- 52.428302765
- ],
- [
- 13.47981739,
- 52.428356171
- ],
- [
- 13.47854805,
- 52.42930603
- ],
- [
- 13.478359222,
- 52.429431915
- ],
- [
- 13.478157997,
- 52.429595947
- ],
- [
- 13.478037834,
- 52.429683685
- ],
- [
- 13.47772789,
- 52.429954529
- ],
- [
- 13.477515221,
- 52.430137634
- ],
- [
- 13.476874352,
- 52.430652618
- ],
- [
- 13.47661972,
- 52.430850983
- ],
- [
- 13.476373672,
- 52.431026459
- ],
- [
- 13.475787163,
- 52.431369781
- ],
- [
- 13.475365639,
- 52.431587219
- ],
- [
- 13.474074364,
- 52.4322052
- ],
- [
- 13.473599434,
- 52.432434082
- ],
- [
- 13.473415375,
- 52.43252182
- ],
- [
- 13.472993851,
- 52.432712555
- ],
- [
- 13.4716959,
- 52.433292389
- ],
- [
- 13.471329689,
- 52.433441162
- ],
- [
- 13.469817162,
- 52.434043884
- ],
- [
- 13.469201088,
- 52.434314728
- ],
- [
- 13.469059944,
- 52.434391022
- ],
- [
- 13.468596458,
- 52.434631348
- ],
- [
- 13.466616631,
- 52.435817719
- ],
- [
- 13.466080666,
- 52.436122894
- ],
- [
- 13.465838432,
- 52.436260223
- ],
- [
- 13.465315819,
- 52.436565399
- ],
- [
- 13.464496613,
- 52.437023163
- ],
- [
- 13.463171005,
- 52.43762207
- ],
- [
- 13.462702751,
- 52.437843323
- ],
- [
- 13.462409019,
- 52.437988281
- ],
- [
- 13.46231842,
- 52.438037872
- ],
- [
- 13.462058067,
- 52.438179016
- ],
- [
- 13.460422516,
- 52.439193726
- ],
- [
- 13.460037231,
- 52.439479828
- ],
- [
- 13.459775925,
- 52.43970108
- ],
- [
- 13.459723473,
- 52.439754486
- ],
- [
- 13.459409714,
- 52.440021515
- ],
- [
- 13.459005356,
- 52.440368652
- ],
- [
- 13.458240509,
- 52.441017151
- ],
- [
- 13.457676888,
- 52.441509247
- ],
- [
- 13.456965446,
- 52.442108154
- ],
- [
- 13.456719398,
- 52.442359924
- ],
- [
- 13.456583023,
- 52.442497253
- ],
- [
- 13.456512451,
- 52.442592621
- ],
- [
- 13.456332207,
- 52.442821503
- ],
- [
- 13.456071854,
- 52.443252563
- ],
- [
- 13.455370903,
- 52.444972992
- ],
- [
- 13.455272675,
- 52.44519043
- ],
- [
- 13.455172539,
- 52.4454422
- ],
- [
- 13.454929352,
- 52.445533752
- ],
- [
- 13.454372406,
- 52.445556641
- ],
- [
- 13.452836037,
- 52.44562149
- ],
- [
- 13.451435089,
- 52.445671082
- ],
- [
- 13.449950218,
- 52.445732117
- ],
- [
- 13.449712753,
- 52.445739746
- ],
- [
- 13.449320793,
- 52.44575882
- ],
- [
- 13.448624611,
- 52.445781708
- ],
- [
- 13.448477745,
- 52.445789337
- ],
- [
- 13.447191238,
- 52.445838928
- ],
- [
- 13.445914268,
- 52.445884705
- ],
- [
- 13.445550919,
- 52.445896149
- ],
- [
- 13.444639206,
- 52.445934296
- ],
- [
- 13.444497108,
- 52.445941925
- ],
- [
- 13.444350243,
- 52.445953369
- ],
- [
- 13.4439888,
- 52.44600296
- ],
- [
- 13.442544937,
- 52.446834564
- ],
- [
- 13.441972733,
- 52.447113037
- ],
- [
- 13.440879822,
- 52.447547913
- ],
- [
- 13.440397263,
- 52.447731018
- ],
- [
- 13.440110207,
- 52.447826385
- ],
- [
- 13.439696312,
- 52.447929382
- ],
- [
- 13.439285278,
- 52.448001862
- ],
- [
- 13.439059258,
- 52.448059082
- ],
- [
- 13.438909531,
- 52.448108673
- ],
- [
- 13.438746452,
- 52.448192596
- ],
- [
- 13.438674927,
- 52.448131561
- ],
- [
- 13.437895775,
- 52.447433472
- ],
- [
- 13.437681198,
- 52.447235107
- ],
- [
- 13.437045097,
- 52.446689606
- ],
- [
- 13.436873436,
- 52.44651413
- ],
- [
- 13.436362267,
- 52.446037292
- ],
- [
- 13.436190605,
- 52.445854187
- ],
- [
- 13.436362267,
- 52.446037292
- ],
- [
- 13.436873436,
- 52.44651413
- ],
- [
- 13.437045097,
- 52.446689606
- ],
- [
- 13.437681198,
- 52.447235107
- ],
- [
- 13.437660217,
- 52.447479248
- ],
- [
- 13.437644005,
- 52.447570801
- ],
- [
- 13.437449455,
- 52.448303223
- ],
- [
- 13.437498093,
- 52.448410034
- ],
- [
- 13.437361717,
- 52.448432922
- ],
- [
- 13.437185287,
- 52.448471069
- ],
- [
- 13.437046051,
- 52.448516846
- ],
- [
- 13.436974525,
- 52.448551178
- ],
- [
- 13.436769485,
- 52.448654175
- ],
- [
- 13.436707497,
- 52.448688507
- ],
- [
- 13.436658859,
- 52.448745728
- ],
- [
- 13.436600685,
- 52.44877243
- ],
- [
- 13.436264038,
- 52.448947906
- ],
- [
- 13.436333656,
- 52.449050903
- ],
- [
- 13.436501503,
- 52.449287415
- ],
- [
- 13.436527252,
- 52.449321747
- ],
- [
- 13.436580658,
- 52.449398041
- ],
- [
- 13.438248634,
- 52.451702118
- ],
- [
- 13.438462257,
- 52.452030182
- ],
- [
- 13.438505173,
- 52.452144623
- ],
- [
- 13.43860054,
- 52.452201843
- ],
- [
- 13.438648224,
- 52.452415466
- ],
- [
- 13.438677788,
- 52.452579498
- ],
- [
- 13.438673973,
- 52.452697754
- ],
- [
- 13.438651085,
- 52.452781677
- ],
- [
- 13.438504219,
- 52.453048706
- ],
- [
- 13.438376427,
- 52.453136444
- ],
- [
- 13.438240051,
- 52.453483582
- ],
- [
- 13.437959671,
- 52.454143524
- ],
- [
- 13.437572479,
- 52.455108643
- ],
- [
- 13.43737793,
- 52.455593109
- ],
- [
- 13.437252045,
- 52.455905914
- ],
- [
- 13.437194824,
- 52.456039429
- ],
- [
- 13.437150002,
- 52.456150055
- ],
- [
- 13.436961174,
- 52.456607819
- ],
- [
- 13.436709404,
- 52.457176208
- ],
- [
- 13.436512947,
- 52.45759201
- ],
- [
- 13.436303139,
- 52.457977295
- ],
- [
- 13.436096191,
- 52.458377838
- ],
- [
- 13.436008453,
- 52.458568573
- ],
- [
- 13.435801506,
- 52.459049225
- ],
- [
- 13.435427666,
- 52.459831238
- ],
- [
- 13.435299873,
- 52.46018219
- ],
- [
- 13.435123444,
- 52.460720062
- ],
- [
- 13.435070038,
- 52.460891724
- ],
- [
- 13.435009956,
- 52.461063385
- ],
- [
- 13.434931755,
- 52.461303711
- ],
- [
- 13.434843063,
- 52.46149826
- ],
- [
- 13.434798241,
- 52.461856842
- ],
- [
- 13.434743881,
- 52.462051392
- ],
- [
- 13.434661865,
- 52.462287903
- ],
- [
- 13.434613228,
- 52.462429047
- ],
- [
- 13.434565544,
- 52.462532043
- ],
- [
- 13.434479713,
- 52.462696075
- ],
- [
- 13.434398651,
- 52.462814331
- ],
- [
- 13.434216499,
- 52.462982178
- ],
- [
- 13.434041023,
- 52.463176727
- ],
- [
- 13.433871269,
- 52.463363647
- ],
- [
- 13.433573723,
- 52.4637146
- ],
- [
- 13.433339119,
- 52.464027405
- ],
- [
- 13.432988167,
- 52.46452713
- ],
- [
- 13.432909966,
- 52.46465683
- ],
- [
- 13.433052063,
- 52.464736938
- ],
- [
- 13.433185577,
- 52.464847565
- ],
- [
- 13.434347153,
- 52.46578598
- ],
- [
- 13.43439579,
- 52.465824127
- ],
- [
- 13.434561729,
- 52.465961456
- ],
- [
- 13.4347229,
- 52.466087341
- ],
- [
- 13.435704231,
- 52.466880798
- ],
- [
- 13.436393738,
- 52.466960907
- ],
- [
- 13.436722755,
- 52.466999054
- ],
- [
- 13.436852455,
- 52.467014313
- ],
- [
- 13.438138962,
- 52.467159271
- ],
- [
- 13.440406799,
- 52.467430115
- ],
- [
- 13.441808701,
- 52.467594147
- ],
- [
- 13.441857338,
- 52.467605591
- ],
- [
- 13.442008972,
- 52.467670441
- ],
- [
- 13.44198513,
- 52.467803955
- ],
- [
- 13.441916466,
- 52.468208313
- ],
- [
- 13.441903114,
- 52.468284607
- ],
- [
- 13.441795349,
- 52.46887207
- ],
- [
- 13.441762924,
- 52.468978882
- ],
- [
- 13.44181633,
- 52.469089508
- ],
- [
- 13.441812515,
- 52.469108582
- ],
- [
- 13.441781998,
- 52.469345093
- ],
- [
- 13.441745758,
- 52.469604492
- ],
- [
- 13.441641808,
- 52.469726562
- ],
- [
- 13.441613197,
- 52.469841003
- ],
- [
- 13.441549301,
- 52.470172882
- ],
- [
- 13.441507339,
- 52.470413208
- ],
- [
- 13.441498756,
- 52.470462799
- ],
- [
- 13.441394806,
- 52.471038818
- ],
- [
- 13.441366196,
- 52.471206665
- ],
- [
- 13.441405296,
- 52.471279144
- ],
- [
- 13.441369057,
- 52.471508026
- ],
- [
- 13.441458702,
- 52.471515656
- ],
- [
- 13.441511154,
- 52.4715271
- ],
- [
- 13.441576958,
- 52.471569061
- ],
- [
- 13.442343712,
- 52.472263336
- ],
- [
- 13.442426682,
- 52.472301483
- ],
- [
- 13.442523003,
- 52.472324371
- ],
- [
- 13.442516327,
- 52.472366333
- ],
- [
- 13.4425354,
- 52.47240448
- ],
- [
- 13.44308567,
- 52.472888947
- ],
- [
- 13.443483353,
- 52.473255157
- ],
- [
- 13.443502426,
- 52.47328186
- ],
- [
- 13.443569183,
- 52.473423004
- ],
- [
- 13.443605423,
- 52.473564148
- ],
- [
- 13.443622589,
- 52.473712921
- ],
- [
- 13.443624496,
- 52.4737854
- ],
- [
- 13.444513321,
- 52.473873138
- ],
- [
- 13.445295334,
- 52.473960876
- ],
- [
- 13.445151329,
- 52.474128723
- ],
- [
- 13.445291519,
- 52.474163055
- ],
- [
- 13.446117401,
- 52.474380493
- ],
- [
- 13.446929932,
- 52.474597931
- ],
- [
- 13.44698143,
- 52.47460556
- ],
- [
- 13.447024345,
- 52.47460556
- ],
- [
- 13.447067261,
- 52.474601746
- ],
- [
- 13.447439194,
- 52.475803375
- ],
- [
- 13.447550774,
- 52.476112366
- ],
- [
- 13.447616577,
- 52.476333618
- ],
- [
- 13.447663307,
- 52.476425171
- ],
- [
- 13.44804287,
- 52.477684021
- ],
- [
- 13.448085785,
- 52.47782135
- ],
- [
- 13.448119164,
- 52.477935791
- ],
- [
- 13.448202133,
- 52.478160858
- ],
- [
- 13.448659897,
- 52.478504181
- ],
- [
- 13.449481964,
- 52.479129791
- ],
- [
- 13.450310707,
- 52.479755402
- ],
- [
- 13.450602531,
- 52.479976654
- ],
- [
- 13.450725555,
- 52.480072021
- ],
- [
- 13.451802254,
- 52.480880737
- ],
- [
- 13.452273369,
- 52.481243134
- ],
- [
- 13.45246315,
- 52.481388092
- ],
- [
- 13.452612877,
- 52.481498718
- ],
- [
- 13.45324707,
- 52.48197937
- ],
- [
- 13.453961372,
- 52.482517242
- ],
- [
- 13.456096649,
- 52.484138489
- ],
- [
- 13.456254959,
- 52.484260559
- ],
- [
- 13.457948685,
- 52.485546112
- ],
- [
- 13.458347321,
- 52.485839844
- ],
- [
- 13.456992149,
- 52.486808777
- ],
- [
- 13.456432343,
- 52.487201691
- ],
- [
- 13.455653191,
- 52.487758636
- ],
- [
- 13.453977585,
- 52.488941193
- ],
- [
- 13.454572678,
- 52.489376068
- ],
- [
- 13.455964088,
- 52.490432739
- ],
- [
- 13.456983566,
- 52.491188049
- ],
- [
- 13.457713127,
- 52.491714478
- ],
- [
- 13.457920074,
- 52.491786957
- ],
- [
- 13.458016396,
- 52.491840363
- ],
- [
- 13.45939827,
- 52.492835999
- ],
- [
- 13.459500313,
- 52.492912292
- ],
- [
- 13.460221291,
- 52.493465424
- ],
- [
- 13.460617065,
- 52.493797302
- ],
- [
- 13.460804939,
- 52.493942261
- ],
- [
- 13.461070061,
- 52.494159698
- ],
- [
- 13.461850166,
- 52.494800568
- ],
- [
- 13.462110519,
- 52.49521637
- ],
- [
- 13.462246895,
- 52.495323181
- ],
- [
- 13.46389389,
- 52.496665955
- ],
- [
- 13.464204788,
- 52.496753693
- ],
- [
- 13.464544296,
- 52.497119904
- ],
- [
- 13.464606285,
- 52.497215271
- ],
- [
- 13.464643478,
- 52.497310638
- ],
- [
- 13.464666367,
- 52.497421265
- ],
- [
- 13.464753151,
- 52.497566223
- ],
- [
- 13.464924812,
- 52.497829437
- ],
- [
- 13.465081215,
- 52.49805069
- ],
- [
- 13.465513229,
- 52.498714447
- ],
- [
- 13.465647697,
- 52.49892807
- ],
- [
- 13.465722084,
- 52.499099731
- ],
- [
- 13.465786934,
- 52.499248505
- ],
- [
- 13.465964317,
- 52.499668121
- ],
- [
- 13.466071129,
- 52.499950409
- ],
- [
- 13.466582298,
- 52.501167297
- ],
- [
- 13.466583252,
- 52.501277924
- ],
- [
- 13.466501236,
- 52.501941681
- ],
- [
- 13.466501236,
- 52.502063751
- ],
- [
- 13.466513634,
- 52.502140045
- ],
- [
- 13.466565132,
- 52.502277374
- ],
- [
- 13.466641426,
- 52.502422333
- ],
- [
- 13.466691971,
- 52.502490997
- ],
- [
- 13.466762543,
- 52.502563477
- ],
- [
- 13.466827393,
- 52.502609253
- ],
- [
- 13.466879845,
- 52.502632141
- ],
- [
- 13.466954231,
- 52.502655029
- ],
- [
- 13.467047691,
- 52.502666473
- ],
- [
- 13.467115402,
- 52.502670288
- ],
- [
- 13.467172623,
- 52.502670288
- ],
- [
- 13.467508316,
- 52.502624512
- ],
- [
- 13.468510628,
- 52.502464294
- ],
- [
- 13.469779968,
- 52.502254486
- ],
- [
- 13.469991684,
- 52.502220154
- ],
- [
- 13.471398354,
- 52.502017975
- ],
- [
- 13.474914551,
- 52.501502991
- ],
- [
- 13.475172043,
- 52.5014534
- ],
- [
- 13.475209236,
- 52.501560211
- ],
- [
- 13.475388527,
- 52.501979828
- ],
- [
- 13.477026939,
- 52.50170517
- ],
- [
- 13.477138519,
- 52.50169754
- ],
- [
- 13.477298737,
- 52.50170517
- ],
- [
- 13.477692604,
- 52.501735687
- ],
- [
- 13.478367805,
- 52.501785278
- ],
- [
- 13.48141098,
- 52.502010345
- ],
- [
- 13.483391762,
- 52.502151489
- ],
- [
- 13.484308243,
- 52.502223969
- ],
- [
- 13.484597206,
- 52.502246857
- ],
- [
- 13.486724854,
- 52.502410889
- ],
- [
- 13.487017632,
- 52.502426147
- ],
- [
- 13.487155914,
- 52.502399445
- ],
- [
- 13.488366127,
- 52.502124786
- ],
- [
- 13.48913765,
- 52.501945496
- ],
- [
- 13.490619659,
- 52.501483917
- ],
- [
- 13.490981102,
- 52.501377106
- ],
- [
- 13.492963791,
- 52.500965118
- ],
- [
- 13.493370056,
- 52.500881195
- ],
- [
- 13.494781494,
- 52.500595093
- ],
- [
- 13.495025635,
- 52.500537872
- ],
- [
- 13.495450974,
- 52.500431061
- ],
- [
- 13.495686531,
- 52.500354767
- ],
- [
- 13.495876312,
- 52.500293732
- ],
- [
- 13.496304512,
- 52.500156403
- ],
- [
- 13.497889519,
- 52.499641418
- ]
- ]
- }
- }
- }
- },
- "aOutboundAllDay20190301ArrDep": {
- "arrivalDepartureId": "YS1vdXRib3VuZC1hbGwtZGF5:MjAxOS0wMy0wMQ==:Mw==:LTE=:LTE=",
- "tripId": "a-outbound-all-day",
- "date": "2019-03-01T00:00:00",
- "stopSequence": 3,
- "frequenciesRow": -1,
- "frequenciesIt": -1
- },
- "bDowntownOnWorkingDays20190608ArrDep": {
- "arrivalDepartureId": "Yi1kb3dudG93bi1vbi13b3JraW5nLWRheXM=:MjAxOS0wMy0wOA==:Mw==:MQ==:Mg==",
- "tripId": "b-downtown-on-working-days",
- "date": "2019-03-08T00:00:00",
- "stopSequence": 3,
- "frequenciesRow": 1,
- "frequenciesIt": 2
- },
- "aOutboundAllDay20190301Con": {
- "connectionId": "YS1vdXRib3VuZC1hbGwtZGF5:MjAxOS0wMy0wMQ==:Mw==:LTE=:LTE=",
- "tripId": "a-outbound-all-day",
- "date": "2019-03-01T00:00:00",
- "fromStopSequence": 3,
- "frequenciesRow": -1,
- "frequenciesIt": -1
- },
- "bDowntownOnWorkingDays20190608Con": {
- "connectionId": "Yi1kb3dudG93bi1vbi13b3JraW5nLWRheXM=:MjAxOS0wMy0wOA==:Mw==:MQ==:Mg==",
- "tripId": "b-downtown-on-working-days",
- "date": "2019-03-08T00:00:00",
- "fromStopSequence": 3,
- "frequenciesRow": 1,
- "frequenciesIt": 2
- }
- }
-}
diff --git a/test/sample-gtfs-feed.sh b/test/sample-gtfs-feed.sh
index a368ee6..fe136a4 100755
--- a/test/sample-gtfs-feed.sh
+++ b/test/sample-gtfs-feed.sh
@@ -8,11 +8,15 @@ set -x
env | grep '^PG' || true
-psql -c 'create database sample_gtfs_feed'
-export PGDATABASE='sample_gtfs_feed'
+# path_to_db="sample-gtfs-feed.duckdb"
+path_to_db="$(mktemp -d)/sample-gtfs-feed.duckdb"
+# path_to_db=':memory:'
+# todo: what about sample-gtfs-feed@0.13?
# --lower-case-lang-codes: Even though sample-gtfs-feed@0.11.2 *does not* contain invalid-case language codes (e.g. de_aT or de-at), we check that with --lower-case-lang-codes valid ones are still accepted.
../cli.js -d --trips-without-shape-id --lower-case-lang-codes -- \
+ "$path_to_db" \
+ ../node_modules/sample-gtfs-feed/gtfs/feed_info.txt \
../node_modules/sample-gtfs-feed/gtfs/agency.txt \
../node_modules/sample-gtfs-feed/gtfs/calendar.txt \
../node_modules/sample-gtfs-feed/gtfs/calendar_dates.txt \
@@ -23,8 +27,7 @@ export PGDATABASE='sample_gtfs_feed'
../node_modules/sample-gtfs-feed/gtfs/stop_times.txt \
../node_modules/sample-gtfs-feed/gtfs/levels.txt \
../node_modules/sample-gtfs-feed/gtfs/pathways.txt \
- ../node_modules/sample-gtfs-feed/gtfs/translations.txt \
- | sponge | psql -b
+ ../node_modules/sample-gtfs-feed/gtfs/translations.txt
query=$(cat << EOF
select extract(epoch from t_arrival)::integer as t_arrival
@@ -34,18 +37,19 @@ order by t_arrival
EOF
)
-arr1=$(psql --csv -t -c "$query" | head -n 1)
+arr1=$(duckdb -csv -noheader -c "$query" "$path_to_db" | head -n 1)
if [[ "$arr1" != "1553993700" ]]; then
echo "invalid 1st t_arrival: $arr1" 1>&2
exit 1
fi
-arr2=$(psql --csv -t -c "$query" | head -n 2 | tail -n 1)
+arr2=$(duckdb -csv -noheader -c "$query" "$path_to_db" | head -n 2 | tail -n 1)
if [[ "$arr2" != "1553994180" ]]; then
echo "invalid 2nd t_arrival: $arr2" 1>&2
exit 1
fi
+# In sample-gtfs-feed@0.13, the frequencies-based arrivals/departures are earlier (from 8:00 until 8:59) than the stop_times-based ones (13:13), so across all service days, the earliest departure has to be a frequencies-based one.
arrs_deps_b_downtown_on_working_days=$(cat << EOF
SELECT
stop_sequence,
@@ -58,12 +62,12 @@ arrs_deps_b_downtown_on_working_days=$(cat << EOF
LIMIT 2
EOF
)
-freq_arr_dep1=$(psql --csv -t -c "$arrs_deps_b_downtown_on_working_days" | head -n 1)
+freq_arr_dep1=$(duckdb -csv -noheader -c "$arrs_deps_b_downtown_on_working_days" "$path_to_db" | head -n 1)
if [[ "$freq_arr_dep1" != "1,1552028340,1552028400,1,1" ]]; then
echo "invalid/missing frequencies-based arrival/departure: $freq_arr_dep1" 1>&2
exit 1
fi
-freq_arr_dep2=$(psql --csv -t -c "$arrs_deps_b_downtown_on_working_days" | head -n 2 | tail -n 1)
+freq_arr_dep2=$(duckdb -csv -noheader -c "$arrs_deps_b_downtown_on_working_days" "$path_to_db" | head -n 2 | tail -n 1)
if [[ "$freq_arr_dep2" != "1,1552028640,1552028700,1,2" ]]; then
echo "invalid/missing frequencies-based arrival/departure: $freq_arr_dep2" 1>&2
exit 1
@@ -81,7 +85,7 @@ cons_b_downtown_on_working_days=$(cat << EOF
LIMIT 1
EOF
)
-freq_con1=$(psql --csv -t -c "$cons_b_downtown_on_working_days")
+freq_con1=$(duckdb -csv -noheader -c "$cons_b_downtown_on_working_days" "$path_to_db")
if [[ "$freq_con1" != "1,1552028400,3,1552028760" ]]; then
echo "invalid/missing frequencies-based connection: $freq_con1" 1>&2
exit 1
@@ -93,10 +97,10 @@ connection_during_dst=$(cat << EOF
extract(epoch from t_departure)::integer as dep
FROM connections
WHERE trip_id = 'during-dst-1'
- AND t_departure = '2019-03-31T01:58+01'
+ AND t_departure = '2019-03-31T01:58:00+01:00'
EOF
)
-dst1=$(psql --csv -t -c "$connection_during_dst" | head -n 1)
+dst1=$(duckdb -csv -noheader -c "$connection_during_dst" "$path_to_db" | head -n 1)
if [[ "$dst1" != "0,1553993880" ]]; then
echo "invalid/missing DST t_departure: $dst1" 1>&2
exit 1
@@ -113,8 +117,8 @@ airport_levels=$(cat << EOF
LIMIT 1
EOF
)
-lvl1=$(psql --csv -t -c "$airport_levels" | head -n 1)
-if [[ "$lvl1" != "airport-level-0,0,ground level" ]]; then
+lvl1=$(duckdb -csv -noheader -c "$airport_levels" "$path_to_db" | head -n 1)
+if [[ "$lvl1" != 'airport-level-0,0.0,ground level' ]]; then
echo "invalid/missing lowest airport-% level: $lvl1" 1>&2
exit 1
fi
@@ -129,8 +133,8 @@ airportPathway=$(cat << EOF
LIMIT 1
EOF
)
-pw1=$(psql --csv -t -c "$airportPathway" | head -n 1)
-if [[ "$pw1" != "escalator,f" ]]; then
+pw1=$(duckdb -csv -noheader -c "$airportPathway" "$path_to_db" | head -n 1)
+if [[ "$pw1" != 'escalator,false' ]]; then
echo "invalid/missing DST t_departure: $pw1" 1>&2
exit 1
fi
@@ -143,7 +147,7 @@ timepoint_exact=$(cat << EOF
LIMIT 1
EOF
)
-exact1=$(psql --csv -t -c "$timepoint_exact" | head -n 1)
+exact1=$(duckdb -csv -noheader -c "$timepoint_exact" "$path_to_db" | head -n 1)
if [[ "$exact1" != "exact" ]]; then
echo "invalid/missing DST t_departure: $exact1" 1>&2
exit 1
@@ -157,7 +161,7 @@ stops_translations=$(cat << EOF
AND record_id = 'airport-entrance'
EOF
)
-airport_entrance_translation=$(psql --csv -t -c "$stops_translations")
+airport_entrance_translation=$(duckdb -csv -noheader -c "$stops_translations" "$path_to_db")
if [[ "$airport_entrance_translation" != "Eingang,de-DE" ]]; then
echo "invalid/missing stop translation: $airport_entrance_translation" 1>&2
exit 1
@@ -173,7 +177,7 @@ stops_translated=$(cat << EOF
AND stop_id = 'airport-entrance'
EOF
)
-translated_airport_entrance=$(psql --csv -t -c "$stops_translated")
+translated_airport_entrance=$(duckdb -csv -noheader -c "$stops_translated" "$path_to_db")
if [[ "$translated_airport_entrance" != "airport-entrance,Eingang,de-DE" ]]; then
echo "invalid/missing translated stop: $translated_airport_entrance" 1>&2
exit 1
@@ -187,10 +191,10 @@ WHERE route_id = ANY(ARRAY['A', 'B'])
ORDER BY trip_id
EOF
)
-wheelchair_accessible_arrs_deps_rows="$(psql --csv -t -c "$wheelchair_accessible_arrs_deps_query")"
+wheelchair_accessible_arrs_deps_rows="$(duckdb -csv -noheader -c "$wheelchair_accessible_arrs_deps_query" "$path_to_db")"
wheelchair_accessible_arrs_deps_expected=$(cat << EOF
-a-downtown-all-day,
-a-outbound-all-day,
+a-downtown-all-day,NULL
+a-outbound-all-day,NULL
b-downtown-on-weekends,accessible
b-downtown-on-working-days,accessible
b-outbound-on-weekends,unknown
@@ -210,10 +214,10 @@ WHERE route_id = ANY(ARRAY['A', 'B'])
ORDER BY trip_id
EOF
)
-bikes_allowed_arrs_deps_rows="$(psql --csv -t -c "$bikes_allowed_arrs_deps_query")"
+bikes_allowed_arrs_deps_rows="$(duckdb -csv -noheader -c "$bikes_allowed_arrs_deps_query" "$path_to_db")"
bikes_allowed_arrs_deps_expected=$(cat << EOF
-a-downtown-all-day,
-a-outbound-all-day,
+a-downtown-all-day,NULL
+a-outbound-all-day,NULL
b-downtown-on-weekends,unknown
b-downtown-on-working-days,unknown
b-outbound-on-weekends,allowed
@@ -229,9 +233,10 @@ frequencies_it_query=$(cat << EOF
SELECT t_departure, stop_sequence, stop_id, frequencies_it
FROM arrivals_departures
WHERE trip_id = 'b-downtown-on-working-days' AND "date" = '2019-05-29' AND frequencies_it = 3
+ORDER BY t_departure
EOF
)
-frequencies_it_rows="$(psql --csv -t -c "$frequencies_it_query")"
+frequencies_it_rows="$(duckdb -csv -noheader -c "$frequencies_it_query" "$path_to_db")"
frequencies_it_expected=$(cat << EOF
2019-05-29 08:10:00+02,1,airport,3
2019-05-29 08:18:00+02,3,lake,3
@@ -253,7 +258,7 @@ ORDER BY t_departure ASC
LIMIT 3
EOF
)
-frequencies_it_connections_rows="$(psql --csv -t -c "$frequencies_it_connections_query")"
+frequencies_it_connections_rows="$(duckdb -csv -noheader -c "$frequencies_it_connections_query" "$path_to_db")"
frequencies_it_connections_expected=$(cat << EOF
1,2019-03-08 08:00:00+01,2019-03-08 08:06:00+01,1
1,2019-03-08 08:05:00+01,2019-03-08 08:11:00+01,2
@@ -273,9 +278,10 @@ SELECT
stop_url, stop_url_lang
FROM stops_translated
WHERE stop_id LIKE 'airport%'
+ORDER BY stop_id, stop_name_lang, stop_desc_lang
EOF
)
-stops_translated_rows="$(psql --csv -t -c "$stops_translated_query")"
+stops_translated_rows="$(duckdb -csv -noheader -nullvalue '' -c "$stops_translated_query" "$path_to_db")"
stops_translated_expected=$(cat << EOF
airport,International Airport (ABC),,train station at the Internationl Airport (ABC),,https://fta.example.org/stations/airport.html,
airport-1,Gleis 1,de-DE,Platform 1,,,
diff --git a/test/stops-without-level-id.sh b/test/stops-without-level-id.sh
index 7431429..473a2b0 100755
--- a/test/stops-without-level-id.sh
+++ b/test/stops-without-level-id.sh
@@ -8,16 +8,16 @@ set -x
shopt -s extglob
-# When omitting levels.txt, --stops-without-level-id/opt.stopsWithoutLevelId should be true by default.
+# Importing should work *without* levels.txt.
# see also https://github.com/public-transport/gtfs-via-postgres/issues/43
../cli.js -d -s -- \
- ../node_modules/sample-gtfs-feed/gtfs/!(levels).txt \
- | grep -c 'stopsWithoutLevelId: true'
+ ':memory:' \
+ ../node_modules/sample-gtfs-feed/gtfs/!(levels).txt
# Importing should work *with* --stops-without-level-id (and without levels.txt).
# see also https://github.com/public-transport/gtfs-via-postgres/issues/43#issuecomment-1632657546
../cli.js -d -s --stops-without-level-id -- \
- ../node_modules/sample-gtfs-feed/gtfs/!(levels).txt \
- >/dev/null
+ ':memory:' \
+ ../node_modules/sample-gtfs-feed/gtfs/!(levels).txt
echo 'works ✔'