Selenium is great to automate repetitive task in a web browser. We can take that concept one step further and use it to create end-to-end tests for web applications. Today we look at an approach to keep the initialisation code for Selenium in one place and use it in as many tests as we need.
The Jetpack extension of WordPress.com uses a lot of JavaScript to work. With Selenium we do not have to bother about that and let our web browser deal with it. This allows us to grab the statistics Jetpack collects for this site.
Last week we covered the basic parts to automate our browser with Selenium. If you try that with real web pages, you quickly notice that they are often not enough. In this post we look at few helpful tricks to make your Selenium automation more robust.
With Selenium ready and a solution in place to download drivers with ease, we now can start with the interesting part and automate our browser.
Downloading drivers for Selenium is annoying. We need to find the right driver for the specific browser version that runs on our computer. Luckily for us, there is a project that can do this work for us.
The more JavaScript a web site has, the more makes it sense to access it with a web browser. Over the next weeks we explore how Selenium can help us to automate a web browser to get the data from a web page we are interested in.
We must renew TLS/SSL certificates before they reach their end of life. Otherwise, our users get an unfriendly message about an unsecure web page. Why not write a Python script that checks our sites and shows how many days are remaining for our certificates?
With requests, Beautiful Soup, and the ultimate sitemap parser we have everything together to create our own basic link checker. Let us combine the three parts to something useful and check if our web site has broken links.
Sitemaps are great starting points if you need to find a list of all pages in a web site. Let us look at a little helper package that allows us to read the different sitemap formats with ease.
Getting HTML pages into our Python application is a straightforward task, thanks to requests. But what do we do now? Let us look how Beautiful Soup can help us getting the data out of the often messed up HTML code.