Parallel requests for query and diffs
-
utils.add_osm_data(df, path=f'{../../datasets/merged/batches_with_way_ids/unfaelle_berlin_{index}_osm.csv', radius=5)
can be called like before but instead of sync request it initiates 10 tcp handshakes at a time (keep-alive) and then proceeds to do 10 http request / tcp connection. - For diff query:
-
Add all queries up into a list
query_list = [] for element in group_df: adiff = '[adiff:\"2018-01-01T00:00:00Z\",\"2021-12-31T00:00:00Z\"]' query_list.append({'params':{'data':f'{adiff}[out:xml];way({element});out meta;'}})
-
Get your respones collected:
responses = asyncio.run(utils.overpass_query_parallel_text(query_list))
Notice
overpass_query_parallel_text
collects all request that have not received a response with a status code of200
and retries them in another tcp connection. It only returns if all responses been successful - otherwise a timeout will occur.
-