scraping
Building offline archives
Intro I’ve been looking into some ways to work offline. Here are some reasons for that: Decrease in the quality of general purpose search engine results More targetted searches Better response times and decreased latency for slow websites (since after I download them they’re served from my local network, maybe directly from the disk of my laptop) Sites are disappearing at a high rate.
Polygon gridding using Geopandas and Shapely
Intro This post will discuss some work involving maps I’ve helped a client with. The main goal of the project was collecting various datasets from web services. One of those web services has an endpoint that receives as a parameter a series of points that define a polygon for which the API request is made (the response will be a series of resources that are located inside that polygon). The API supports pagination, so if the area of the polygon is too big, we’ll have to do additional requests for all the result pages.