#270: Persisting Data in DuckDB
Last week we made our first steps with DuckDB and its in-memory capabilities. In this post we persist data with DuckDB so that we can reuse it between our scripts.
Blog
#269: Introducing DuckDB
DuckDB is an in-process SQL OLAP (Online Analytical Processing) database designed for efficient analytical queries. Known as the "SQLite for analytics", it integrates seamlessly with Python, making it ideal for data analysis. Let us explore the benefits it can bring to our tool box. We focus on the Python integration, but you can use DuckDB with many more programming languages.
#268: Manage Projects With uv
Last week we explored the uv tool and saw how much of pip we can reuse while profiting from the massive speed boost. In this post we explore the final part of uv and see how it can help us to manage our projects and prepare our package for pypi.org.
#267: uv - The Fast Pip Replacement
If we dive into machine learning, we quickly run into a problem: Incompatible package versions. We can solve that problem with virtual environments, but that brings us right to the next one. Pip is a great tool, but when we need to install the same package multiple times, pip will download the package again and again. That is slow and breaks the flow. Let us explore an alternative solution.
#266: Using Parquet Files in Pandas
In last week’s post we explored the Parquet format and how we can work with it using pyarrow and fastparquet. Now it is time to find out how we can use Parquet files with Pandas so that we can profit from this storage efficient format in our daily work.
#265: Working With Parquet Files
Now that we know how to create a large amount of test data with Faker, we should find an efficient way to store the data. Most developers know CSV files, but is there a more efficient format we can use? On my search to find an answer to this question, the Parquet format showed up and it sounds like the tool for this task. Let us find out if this is the case and how we can use it.
In this post we use pyarrow and fastparquet to work with Parquet files, while Pandas will be the topic of the next post.
#264: Custom Providers for Faker
Faker is a test data generator we explored in the last post. Despite the large list of options to generate data, sometimes we need data in a format that Faker cannot offer. Should that happen, we can create our own data provider. Let us figure out how to do that.
#263: Generate Realistic Test Data With Faker
If you have ever worked on a project that needed realistic test data, you know how tedious it can be to create it manually. Hardcoding names, addresses, or even phone numbers into your test suite not only clutters your code but also makes it inflexible. Thankfully, the Python Faker library comes to the rescue.
#262: When () Disappear in Method Calls
Have you ever run into unexpected bugs when checking conditions in your Python code? If so, you might have stumbled upon a subtle, yet common mistake: using a method reference without parentheses when evaluating conditions.
#261: The Counter Class
When working with data in Python, we often need to count occurrences of items in a list or a string. While we could write our own code to do this, Python comes with the collections module that provides a convenient and efficient way to handle this task: the Counter class.