With requests, Beautiful Soup, and the ultimate sitemap parser we have everything together to create our own basic link checker. Let us combine the three parts to something useful and check if our web site has broken links.
Sitemaps are great starting points if you need to find a list of all pages in a web site. Let us look at a little helper package that allows us to read the different sitemap formats with ease.
Getting HTML pages into our Python application is a straightforward task, thanks to requests. But what do we do now? Let us look how Beautiful Soup can help us getting the data out of the often messed up HTML code.
Web crawlers are still a hot topic. More and more data is on the web; unfortunately, often not in the form we need it. Today we look at requests, the elegant HTTP library for Python that gives us a good starting point to interact with web applications.
With the basics of date and time covered, we can look at how the dateutil module can help us with relative dates.
After a deeper look at date last week, we explore in this post the datetime and time objects of Python.
I waited a long time to address date and time because that topic has the potential to get complicated quickly. In a side project I needed some date arithmetic and that is a good opportunity to look at date and time in Python.
I needed to return multiple values from a function. While Python allows that with tuples, it is a lot of magic involved and the meaning in the order of values in the tuple is nowhere written down. Is there a better way to create a data transfer object without much effort? Let's find out.
Your terminal applications do not need to have a boring output. Today we take a look at how we can create nice looking tables without much effort.
Since I concluded my experiments with Tweepy in April, a new and important Twitter endpoint finally arrived: Bookmarks. Let's have a look how we can access our Twitter bookmarks with Tweepy.