The Basics: What These Commands Do

Before we dive into the details, let's quickly outline what these commands do:

1. docker-compose up -d: Starts containers in detached mode

2. docker-compose up --build: Rebuilds images before starting containers

Now, let's break down each command and explore when to use them.

docker-compose up -d: Quick Starts in the Background

The -d flag in docker-compose up -d stands for "detached" mode. Here's what you need to know:

• Background Operation: This command starts your containers in the background, freeing up your terminal for other tasks.

• Uses Existing Images: It uses pre-built images if they're available locally.

• Pulls if Necessary: If an image isn't found locally, it pulls from the registry.

• No Rebuilding: It doesn't rebuild images, even if their Dockerfiles have changed.

When to Use It:

• When you want to quickly start your services without any changes

• For production environments where images are pre-built

• When you need to run services in the background

docker-compose up --build: Ensuring Fresh Builds

The --build flag forces Docker Compose to rebuild your images. Key points:

• Forced Rebuild: It rebuilds all images defined with a build context in your docker-compose.yml file.

• Foreground Operation: By default, this runs in the foreground (attached mode).

• Incorporates Changes: Ensures your containers use the latest version of your code and Dockerfile instructions.

When to Use It:

• After making changes to your Dockerfile

• When you've updated your application code

• In development environments where you frequently iterate on your containers

Combining the Commands

You're not limited to using these flags separately. You can combine them for different effects:

docker-compose up -d --build

This command rebuilds your images and then starts the containers in detached mode. It's useful when you want to ensure you're running the latest version of your services in the background.

Best Practices

1. Development Workflows: Use --build frequently in development to ensure you're testing the latest changes.

2. CI/CD Pipelines: Incorporate --build in your continuous integration processes to catch build issues early.

3. Production Deployments: Use -d for production deployments where images are pre-built and validated.

4. Documentation: Clearly document in your project README which command developers should use and when.

Conclusion

Understanding the nuances between docker-compose up -d and docker-compose up --build can significantly improve your Docker workflow. By choosing the right command for your situation, you can save time, ensure consistency, and avoid potential issues caused by outdated images.

Remember, the key is to use -d when you need quick, background starts of existing services, and --build when you need to incorporate recent changes into your containers.

What's your experience with these Docker Compose commands? Do you have any tips or tricks to share? Let us know in the comments below!

Understanding the "No Space Left on Device" Error in Docker

The "No Space Left on Device" error in Docker indicates that the filesystem where Docker stores its data has run out of free space. Docker uses disk space to store images, containers, volumes, and other data in the /var/lib/docker directory by default. When this directory fills up, Docker operations—such as pulling images, creating containers, or writing to volumes—fail.

Docker's space usage can quickly balloon due to:

• Accumulated unused images and containers

• Orphaned volumes

• Build cache from image creation

• Logs and other operational data

This error significantly impacts Docker operations, preventing you from creating new containers, pulling images, or even running existing containers in some cases.

Quick Fix: Freeing Up Space in Docker

To quickly resolve the "No Space Left on Device" error, use these commands to clean up Docker resources:

1. Remove unused Docker objects:

docker system prune

This command removes all stopped containers, unused networks, dangling images, and build cache.

2. Remove unused images:

docker image prune

3. Clean up volumes:

docker volume prune

After running these commands, restart the Docker service:

sudo systemctl restart docker

Monitoring Docker Disk Usage

To prevent future space issues, regularly monitor your Docker disk usage:

1. Check overall disk space:

df -h

2. View Docker-specific usage:

docker system df

Set up alerts for low disk space and implement a regular maintenance schedule to keep your Docker environment healthy.

Root Causes of Docker Space Issues

Understanding the root causes helps in long-term prevention:

1. Accumulation of unused resources: Over time, unused images, containers, and volumes consume significant space.

2. Large or numerous Docker volumes: Data-heavy applications can quickly fill up volumes.

3. Default storage location limitations: The partition containing /var/lib/docker may be too small.

4. System-wide disk space shortage: Your entire system might be running low on space, affecting Docker operations.

Advanced Solutions for Docker Space Management

For more robust space management:

1. Configure a different storage driver: Some drivers are more space-efficient. Research options like overlay2 or devicemapper.

2. Move Docker's data directory: Relocate to a larger partition:

    # Stop Docker
    sudo systemctl stop docker
   
    # Move the data
    sudo mv /var/lib/docker /path/to/new/location
   
    # Update Docker's configuration
    sudo nano /etc/docker/daemon.json
    # Add: {"data-root": "/path/to/new/location"}
   
    # Restart Docker
    sudo systemctl start docker
   

3. Implement image lifecycle policies: Automatically remove old or unused images based on age or usage patterns.

4. Use Docker Compose: Better manage resources across multiple containers and services.

Optimizing Dockerfiles for Space Efficiency

Efficient Dockerfiles lead to smaller images and reduced space usage:

1. Use multi-stage builds: Separate build-time dependencies from runtime dependencies.

2. Minimize layer size:

   • Combine RUN commands

   • Clean up in the same layer where files are added

3. Leverage .dockerignore: Exclude unnecessary files from the build context.

4. Choose appropriate base images: Use slim or alpine versions when possible.

Preventing Future "No Space Left" Errors

Implement these strategies to avoid recurring space issues:

1. Set up regular cleanup cron jobs:

    0 0 * * * docker system prune -af --volumes
   

2. Use disk space monitoring tools: Consider tools like Prometheus with node_exporter for comprehensive monitoring.

3. Educate team members: Share Docker best practices across your development team.

4. Consider container orchestration: Solutions like Kubernetes can help with resource allocation and management.

Key Takeaways

• The "No Space Left on Device" error often stems from Docker's storage management.

• Regular cleanup and monitoring are crucial for preventing space issues.

• Advanced solutions involve reconfiguring Docker's storage setup.

• Optimizing Dockerfiles and implementing best practices can significantly reduce space usage.

FAQs

How often should I run Docker cleanup commands?

Run cleanup commands weekly or bi-weekly, depending on your Docker usage. For high-traffic environments, daily cleanups might be necessary.

Can I move Docker's data directory without losing my containers and images?

Yes, if you follow the correct procedure of stopping Docker, moving the data, updating the configuration, and restarting Docker.

What's the difference between docker system prune and docker image prune?

docker system prune removes unused data including stopped containers, unused networks, dangling images, and build cache. docker image prune only removes dangling images.

How do I increase Docker's default storage limit on Windows?

On Windows, you can increase Docker's storage limit through the Docker Desktop settings. Navigate to Settings > Resources > Advanced and adjust the "Disk image size" slider.

Resources

• Docker official documentation on disk space issues

• Advanced Docker storage driver configuration guide

• Best practices for writing Dockerfiles

Why Docker?

Docker has become an essential tool in modern web development. It allows us to create consistent development environments, simplify deployment processes, and ensure that our application runs the same way across different machines. However, running a Next.js application in Docker can sometimes feel like we're sacrificing the rapid feedback loop we're used to in local development.

The Challenge

By default, when you run a Next.js application in a Docker container, changes to your local files aren't automatically reflected in the running application. This is because the container is isolated from your local file system. To bridge this gap and enable hot reloading, we need to make some specific configurations.

The Solution

To enable hot reloading for Next.js in Docker, we need to make changes to three key files: the Dockerfile, the Docker Compose file, and the Next.js configuration file. Let's break down each of these changes.

1. Updating the Dockerfile

First, we need to modify our Dockerfile to run Next.js in development mode:

FROM node:18-alpine

WORKDIR /app

COPY package.json package-lock.json ./
RUN npm ci

COPY . .
COPY next.config.mjs ./next.config.mjs

EXPOSE 3000

CMD ["npm", "run", "dev"]

The key changes here are:

• We're not running npm run build anymore.

• We've changed the CMD to npm run dev to start Next.js in development mode.

2. Modifying the Docker Compose File

Next, we need to update our Docker Compose file to enable volume mounting:

version: '3'
services:
  # ... other services ...

  frontend:
    build:
      context: ./frontend
      dockerfile: Dockerfile
    ports:
      - "3000:3000"
    environment:
      - BACKEND_URL=http://backend:8000
    volumes:
      - ./frontend:/app
      - /app/node_modules
    depends_on:
      - backend

The crucial additions here are:

• We've added volumes to mount the local frontend directory to /app in the container.

• We've included a volume for node_modules to prevent it from being overwritten by the local mount.

3. Adjusting the Next.js Configuration

Finally, we need to modify our Next.js configuration to enable file watching in Docker:

/** @type {import('next').NextConfig} */
const nextConfig = {
  // ... other configurations ...

  webpackDevMiddleware: config => {
    config.watchOptions = {
      poll: 1000,
      aggregateTimeout: 300,
    }
    return config
  },
}

export default nextConfig;

The added webpackDevMiddleware configuration enables polling for file changes, which is necessary for hot reloading to work in Docker.

Putting It All Together

With these changes in place, you can now run your Docker setup with hot reloading enabled. Use the following command to start your containers:

docker-compose up --build

Now, when you make changes to your Next.js application code, you should see those changes reflected almost immediately in your browser, just as if you were running Next.js directly on your host machine.

Conclusion

Enabling hot reloading for Next.js in Docker might seem like a small change, but it can significantly improve your development experience. It combines the consistency and isolation of Docker with the rapid feedback loop we've come to expect in modern web development.

Remember, the first time you run this setup, it might take a while to install all the dependencies. Subsequent runs should be much faster.

By following this guide, you've supercharged your Next.js development environment in Docker. Happy coding!

In this article, we will explore the possibility and benefits of using Grafana without Prometheus, and provide a step-by-step guide on setting up Grafana with MySQL as the data source.

Understanding Grafana

Grafana is an open-source platform for monitoring and observability that excels in visualizing data from a variety of sources. It offers a highly customizable dashboarding experience, allowing users to create, explore, and share dashboards that display data from different systems.

Common Data Sources Supported by Grafana

Grafana supports numerous data sources out-of-the-box, including:

• MySQL

• InfluxDB

• Elasticsearch

• PostgreSQL

• Prometheus

• Graphite

• Cloudwatch

• Azure Monitor

These integrations enable users to visualize data from various databases and services, making Grafana a versatile tool for many applications.

Benefits of Using Grafana Without Prometheus

• Flexibility in Data Source Selection: Grafana's support for multiple data sources means you can choose the one that best fits your needs, whether it's a SQL database, a time-series database, or a cloud service.

• Simplified Architecture: By using Grafana with a data source like MySQL, you can simplify your monitoring architecture, which might be beneficial for specific use cases.

• Cost Savings: Depending on your requirements, using a single data source can reduce complexity and costs associated with managing multiple monitoring tools.

Practical Example: Setting Up Grafana with MySQL

Let's dive into a practical example of setting up Grafana with MySQL as the data source.

Step 1: Install Grafana

Download and install Grafana from the official website. Follow the installation instructions for your operating system.

Step 2: Set Up MySQL and Create a Sample Database

Install MySQL on your system if it's not already installed. Create a sample database and table for storing your data:

CREATE DATABASE sample_db;
USE sample_db;
CREATE TABLE metrics (
  id INT AUTO_INCREMENT PRIMARY KEY,
  metric_name VARCHAR(255) NOT NULL,
  value FLOAT NOT NULL,
  timestamp DATETIME DEFAULT CURRENT_TIMESTAMP
);
INSERT INTO metrics (metric_name, value) VALUES ('cpu_usage', 23.5), ('memory_usage', 55.1);

Step 3: Configure Grafana to Connect to MySQL

• Open Grafana in your browser (default URL is http://localhost:3000).

• Log in with the default credentials (admin/admin, you an read detailed guide to login here).

• Navigate to Configuration > Data Sources and click Add data source.

• Select MySQL from the list of data sources.

• Enter the necessary connection details (e.g., host, database name, user, and password) and click Save & Test to verify the connection.

  • Host URL will be likely localhost:3306

  • Database Name: sample_db

  • Add your database username and password in the authentication.

Step 4: Create a Simple Dashboard Using MySQL Data

• Navigate to Create > Dashboard and click Add new panel.

• Select your MySQL data source.

• Write a simple SQL query to fetch data from your sample table:

SELECT
  UNIX_TIMESTAMP(timestamp) as time_sec,
  value as value,
  metric_name as metric
FROM metrics

Customize the visualization as needed and save your dashboard.

Real-World Use Cases

Many organizations leverage Grafana with data sources other than Prometheus to meet specific needs:

• Web Application Monitoring: Using MySQL or PostgreSQL to track user interactions and performance metrics.

• Business Analytics: Visualizing data from SQL databases to monitor KPIs and other business metrics.

• IoT Data Visualization: Utilizing InfluxDB to collect and visualize sensor data.

Conclusion

Grafana is a versatile tool that can be used with a variety of data sources beyond Prometheus. This flexibility allows users to tailor their monitoring and visualization setup to their specific needs, simplifying architecture and potentially reducing costs.

For a deeper comparison between Prometheus and Grafana, you can check out this comprehensive article.

Have you used Grafana with a data source other than Prometheus? Share your experiences and questions in the comments below.

The problem is simple to understand, we are given an integer and we need to convert it into a Roman numeral.

Difficulty: Medium

Roman numerals are represented by seven different symbols: I, V, X, L, C, D, and M.

Conversion Table

For example, 2 is written as II in Roman numeral, just two one's added together. 12 is written as XII, which is simply X + II. The number 27 is written as XXVII, which is XX + V + II.

Roman numerals are usually written from largest to smallest from left to right. However, the numeral for four is not IIII. Instead, the number four is written as IV. Because the one is before the five we subtract it making four. The same principle applies to the number nine, which is written as IX. There are six instances where subtraction is used:

• I can be placed before V (5) and X (10) to make 4 and 9.
• X can be placed before L (50) and C (100) to make 40 and 90.
• C can be placed before D (500) and M (1000) to make 400 and 900.

Given an integer, convert it to a roman numeral.

Example

Example 1:

Input: num = 3
Output: "III"
Explanation: 3 is represented as 3 ones.

Example 2:

Input: num = 58
Output: "LVIII"
Explanation: L = 50, V = 5, III = 3.

Example 3:

Input: num = 1994
Output: "MCMXCIV"
Explanation: M = 1000, CM = 900, XC = 90 and IV = 4.

Constraints

• 1 <= num <= 3999

Solution Template

class Solution:
    def intToRoman(self, num: int) -> str:
        pass  # Write your solution here

We have to implement the intToRoman function that accepts a num integer and returns str.

Finding the Solution

First Thought

On reading the question, the first thing that came to my mind was the solution must involve some repeated division and modulus. Why? Check out the following figure:

Let's understand, what does this image means. We have to convert 12 to its Roman numeral equivalent. Firstly, we selected the biggest number that is smaller than 12 and whose conversion is already present in the table given above. That is the number 10. We divided 12 by 10 and the quotient is how many times we need to repeat 10 to get as close to 12 as possible (In our case, it is just 1). Now, we repeat the same process for what remains that is the remainder. Until we got 0, then we add all the strings to get the final roman numeral.

Algorithm:

1. Let roman be the final result we want. Set roman = "".
2. Find the biggest small number in the conversion table than num. Name it base_value.
3. Divide num by base_value.
4. roman += Roman Numeral for Base Value * Quotient.
5. Repeat steps 2 to 4 for the remainder until the remainder becomes 0.

Also make sure that:

• If the number is 0, then directly return the empty string "" (There is no zero in Roman numerals).
• If the number is present in the conversion table, directly return the corresponding value. For example, return X for 10.

Implementing the Recursive Solution

Let's now implement the solution. Start by creating a dictionary for the conversion table:

conversion_table = {
            1:      "I",
            5:      "V",
            10:     "X",
            50:     "L",
            100:    "C",
            500:    "D",
            1000:   "M",
            4:      "IV",
            9:      "IX",
            40:     "XL",
            90:     "XC",
            400:    "CD",
            900:    "CM",
        }

Notice, we are also including the conversions for exceptional numbers like 4, 9, 40, 90, and so on.

Now, handle the base cases:

if num == 0:
    return ""

if num in conversion_table:
    return conversion_table[num]

Now, we will find the base_value, which is the biggest smaller value than the num:

base_value = 0
for value in conversion_table:
    if base_value < value < num:
        base_value = value

Now, just add a recursive call with the formula we constructed before:

return ((num // base_value) * conversion_table[base_value]) + self.intToRoman(num % base_value)

Complete Recursive Solution Code

class Solution:
    def intToRoman(self, num: int) -> str:
        conversion_table = {
            1:      "I",
            5:      "V",
            10:     "X",
            50:     "L",
            100:    "C",
            500:    "D",
            1000:   "M",
            4:      "IV",
            9:      "IX",
            40:     "XL",
            90:     "XC",
            400:    "CD",
            900:    "CM",
        }

        if num == 0:
            return ""

        if num in conversion_table:
            return conversion_table[num]

        # base_value = biggest smaller value than num
        base_value = 0
        for value in conversion_table:
            if base_value < value < num:
                base_value = value

        return ((num // base_value) * conversion_table[base_value]) + self.intToRoman(num % base_value)

Output

>>> Solution().intToRoman(12)
>>> `XII`
>>> Solution().intToRoman(1234)
>>> `MCCXXXIV`

Non-Recursive Solution (Using a While Loop)

Here's a non-recursive solution using the same algorithm:

class Solution:
    def intToRoman(self, num: int) -> str:
        conversion_table = {
            1:      "I",
            5:      "V",
            10:     "X",
            50:     "L",
            100:    "C",
            500:    "D",
            1000:   "M",
            4:      "IV",
            9:      "IX",
            40:     "XL",
            90:     "XC",
            400:    "CD",
            900:    "CM",
        }


        roman = ""    

        while num:

            if num == 0:
                return roman

            if num in conversion_table:
                return roman + conversion_table[num]


            base_value = 0
            for value in conversion_table:
                if base_value < value < num:
                    base_value = value

            roman += ((num // base_value) * conversion_table[base_value])
            num = num % base_value


        return  roman

Result

The solution is accepted on Leetcode:

Try it Out

In this article, we will cover different ways of defining a dictionary, how to access and update values of a dictionary, the dictionary’s built-in methods, and how to use dict comprehension.

We will also learn about restrictions on keys of a dictionary, membership operators, pretty-printing a dictionary, and making a shallow copy vs deep copy of a dictionary.

What is a dictionary in Python?

Python provides several useful data types built-in, Dictionary is one such data structure.

A dictionary is a collection of key: value pairs where each value is accessed using a unique key.

This is in contrast to the list where elements are accessed using their indexes.

We can think of a dictionary as a table with two columns:

Highlights:

• A dictionary is a collection of key: value pairs where each value is accessed using a unique key.

How to define a dictionary?

We can define a dictionary in three different ways. Let's see each one of them by converting the above pictorial representation of the dictionary into a real python one.

Using curly braces {}

We can use the curly braces to define a dictionary by providing key-value pairs separated by a ':' as key: value:

harry_info = {
    "name": "Harry",
    "age": 11,
    "house": "Gryffindor",
}

print(harry_info)
print(type(harry_info))  # <class 'dict'>

Notice that a dictionary can have values of different types.

We can also create an empty dictionary using {} as: empty_dict = {}.

Using dict() function

We can create a dictionary by passing a list of tuples to the dict function:

harry_info = dict([
                   ("name", "Harry"),
                   ("age", 11),
                   ("house", "Gryffindor")
])

print(harry_info)

Using keyword arguments in dict()

We can provide keyword arguments to the dict function when all keys are string:

harry_info = dict(
    name = "Harry",
    age = 11,
    house = "Gryffindor"
)

print(harry_info)

Values in a dictionary

A dictionary can store any type of value including another dictionary. Let's change the harry_info dictionary to contain a nested dictionary house and a list of friends:

harry_info = {
    "name": "Harry",
    "age": 11,
    "house": {
      "name": "Gryffindor",
      "Animal": "Lion",
      "Element": "Fire"   
    },
    "friends": ["Ron", "Hermione"]
}

print(harry_info)

Tip 💡

When we print the above dictionary the output would not be very readable, we can use the pprint function to pretty print a dictionary:

from pprint import pprint

pprint(harry_info)

Output:

{'age': 11,
 'friends': ['Ron', 'Hermione'],
 'house': {'Animal': 'Lion', 'Element': 'Fire', 'name': 'Gryffindor'},
 'name': 'Harry'}

Restrictions on keys

There are two restrictions we need to know when defining keys:

1. Duplicate keys are not allowed

   If we define the same key twice, the second occurrence will override the first one:

    duplicate_dict = {
        "first": 0,
        "second": 2,
        "first":1
    }
   
    print(duplicate_dict)
   

   This will print:

   {'first': 1, 'second': 2} ✅

   and not

   {'first': 0, 'second': 2} ❌

   {'first': 0, 'second': 2, 'first': 1} ❌

2. Keys must be immutable

   Keys must have a type that is immutable, which means any type whose value doesn't change once created like int, bool, str, float, etc.

   That's why we can have a tuple as a key but not a list:

    example_dict = {
        (1, 2): "Tuple",  # Okay to use
        False: "Bool",    # Okay to use
        1: "int",         # Okay to use
        [1, 2]: "list",   # raises TypeError
    }
   

   Using a list as a key raises TypeError: unhashable type: 'list' because it is a mutable object.

   Technically, a key requires a hashable value which means that Python's built-in hash() function should return a unique hash value for that object and all immutable types satisfy this condition.

Highlights:

• We can define a dictionary using:
  • Key-value pairs in curly braces.
  • Using dict function
• A dictionary can have any type of value.
• Key cannot be:
  • Duplicated
  • Mutable (more precisely unhashable)

How to access values in a dictionary?

We can access a value from a dictionary by using the square brackets [].

So dict_name[key] will return the corresponding value.

Let's try to retrieve some values from the harry_info dictionary:

print(harry_info["name"])

Output: "Harry"

print(harry_info["friends"])

Output: ['Ron', 'Hermione']

What if we try to retrieve a value using a key that doesn't exist?

Let's try that too:

print(harry_info["enemies"])

Output: KeyError: 'enemies'

We got a key error which means that we can not retrieve keys that don't exist using the square bracket syntax.

Adding a new key-value pair to the dictionary

As dictionaries are mutable structures, we can add new members by assigning values to new keys using the same square bracket operator:

Syntax: dict_name[new_key] = value

harry_info["enemies"] = ["Voldemort", "Death Eaters"]

Updating existing key-value pair of a dictionary

We can also update the key's values similarly using the square brackets:

Syntax: dict_name[key] = new_value

harry_info["age"] = 17

This will update the age key from 11 to 17.

Using Membership operator with dictionaries

We can use the in and not in operators to check whether a key exists in the dictionary or not:

print('name' in harry_info)     # Output: True

print('name' not in harry_info) # Output: False

print('year' in harry_info)     # Output: False

print('year' not in harry_info) # Output: True

Get the number of key-value pairs

Use the built-in len function to get the number of key-value pairs in a dictionary:

print(len(harry_info))  # Output: 5

Highlights:

• Access values using dict_name[key] syntax.
• Add a new value using dict_name[new_key] = value.
• Update a value using the same syntax as (2).
• Use in and not in operator to check for key existence in a dictionary.
• Use the len function to get the number of key-value pairs.

Built-in Dictionary Methods

get(key, default=None)

This is an alternative to the [] for accessing the values using a key from a dictionary. The difference is that we can provide a default value to return when a key doesn't exist instead of raising a key error.

print(harry_info.get("name"))     # Output: Harry
print(harry_info.get("year"))     # Output: None
print(harry_info.get("year", 7))  # Output: 7

items()

Returns a list of tuples containing the key-value pairs of a dictionary.

print(list(harry_info.items()))

Output:

[('name', 'Harry'), ('age', 17), ('house', {'name': 'Gryffindor', 'Animal': 'Lion', 'Element': 'Fire'}), ('friends', ['Ron', 'Hermione']), ('enemies', ['Voldemort', 'Death Eaters'])]

We can use the items method to loop through a dictionary:

for key, value in harry_info.items():
  print(key, 'is', value)

Output:

name is Harry
age is 17
house is {'name': 'Gryffindor', 'Animal': 'Lion', 'Element': 'Fire'}
friends is ['Ron', 'Hermione']
enemies is ['Voldemort', 'Death Eaters']

keys()

Returns the list of all keys of a dictionary.

print(list(harry_info.keys()))

Output: ['name', 'age', 'house', 'friends', 'enemies']

values()

Returns the list of all values of a dictionary.

print(list(harry_info.values()))

Output: ['Harry', 17, {'name': 'Gryffindor', 'Animal': 'Lion', 'Element': 'Fire'}, ['Ron', 'Hermione'], ['Voldemort', 'Death Eaters']]

pop(key[, default])

Remove the specified key and return the corresponding value.

If the key is not found, d is returned if given, otherwise KeyError is raised.

print(harry_info.pop('enemies'))    # enemies will be removed
print(harry_info.pop('invalid_key', "Invalid Key"))
print(harry_info.pop('invalid_key'))

Output:

['Voldemort', 'Death Eaters']
Invalid Key
KeyError: 'invalid_key'

popitem()

Remove and return the last (key, value) pair as a tuple, but raise KeyError if dictionary is empty.

print(harry_info.popitem())

Output: ('friends', ['Ron', 'Hermione'])

update(iterable)

Merges a dictionary with another dictionary or with an iterable of key-value pairs.

If the dictionary or iterable passed has the same key as the original dictionary then the value of that key is overridden.

When iterable is a dictionary

d1 = {'a': 1, 'b': 2, 'c': 3}
d2 = {'b': 200, 'd': 4}

d1.update(d2)

print(d1)

Output: {'a': 1, 'b': 200, 'c': 3, 'd': 4}

Notice how 'b' is updated and 'd' is added.

When iterable is a list of tuples

d1 = {'a': 1, 'b': 2, 'c': 3}

d1.update([('b', 200), ('d', 4)])

print(d1)

Output: {'a': 1, 'b': 200, 'c': 3, 'd': 4}

We can also provide a keyword arguments to update and the result will be the same:

d1 = {'a': 1, 'b': 2, 'c': 3}

d1.update(b=200, d=4)

print(d1)

Output: {'a': 1, 'b': 200, 'c': 3, 'd': 4}

fromkeys(iterable, value=None)

Create a new dictionary with keys from iterable and all values set to value.

keys = ("key1", "key2", "key3")
example_dict = dict.fromkeys(keys, "default value")
print(example_dict)

Output: {'key1': 'default value', 'key2': 'default value', 'key3': 'default value'}

setdefault(key, default=None)

Return the value for key if key is in the dictionary, otherwise insert that key with a value of default and return default.

# Returns value for 'name'
print(harry_info.setdefault("name"))   
# Returns value for 'name', default is ignored
print(harry_info.setdefault("name", "Harry Potter"))
# Adds key 'last_name' and return its value 'Potter'
print(harry_info.setdefault("last_name", "Potter"))

Output:

Harry
Harry
Potter

copy()

Performs a shallow copy of the dictionary.

harry_copy = harry_info.copy()
print(harry_copy)

Shallow copy vs deep copy

With a shallow copy if the original dictionary contains mutable values they won't be copied, therefore changing the copied dictionary will change the original one.

We can use copy.deepcopy method to perform a deepcopy as shown below:

from copy import deepcopy

d = {
    'number': 1,
    'list': [1, 2]
}

d_shallow = d.copy()
d_shallow['list'].append(3)
print("shallow copy", d_shallow)
print("original dict", d)     # d is also changed

d_deep = deepcopy(d)
d_deep['list'].append(4)
print("deep copy", d_deep)
print("original dict", d)    # d is unchanged

Output:

shallow copy {'number': 1, 'list': [1, 2, 3]}
original dict {'number': 1, 'list': [1, 2, 3]}
deep copy {'number': 1, 'list': [1, 2, 3, 4]}
original dict {'number': 1, 'list': [1, 2, 3]}

clear()

Remove all items from a dictionary

harry_info.clear()
print(harry_info)

Output: {}

Highlights:

• Dictionary provides several methods to access, update, remove, and copy key-value pairs.

Dict Comprehension

You may be familiar with list comprehensions, it is an alternative to for loop and a concise notation to perform some operations for a collection of elements.

You can also write comprehension for a dictionary using the following syntax:

example_dict = {key:value for key, value in iterable}

where the iterable is a list of key-value tuples.

d1 = dict((k, v) for k, v in enumerate("Hello", 1))
print(d1)    # {1: 'H', 2: 'e', 3: 'l', 4: 'l', 5: 'o'}
d2 = {k: v for k, v in enumerate("Bye", 1)}
print(d2)    # {1: 'B', 2: 'y', 3: 'e'}

Enumerate function returns a key: value pair using the iterable passed to it and the start value.

You can learn more about comprehensions in Python in this article: Comprehension in Python: Explained.

Highlights:

• Use dict comprehension to perform operations on key-value pairs of a dictionary in a concise way.

Summary

• Dictionary is a mutable data structure to store key: value pairs.
• Define a dictionary using curly braces or dict function.
• Access or update values of a dictionary using square brackets.
• Dictionary provides built-in methods to add, update, remove, and copy key-value pairs.
• Use dict comprehensions to perform operations on dict in a concise way.

Thanks for reading.

If you found this article helpful, please like and share! Feedbacks are welcome in the comments.

You may also like:

• The Mutable Default Argument Mess in Python
• Comprehensions in Python: Explained
• How to Save User's Data Persistently in Web Browser Using Local Storage
• Linux Commands Reference With Examples

Connect with me on Twitter, GitHub, and LinkedIn.

Prerequisite

You will just need familiarity with basic JavaScript and HTML.

The best way to go through the article is to open your code editor and code along with me.

So, let's start!

What is Local Storage?

LocalStorage is a property of windows object, it allows us to store key-value pairs in a web browser.

The localStorage's data has no expiration date.

The data will not be deleted when the browser is closed and will be available whenever you open the same web page.

Each Protocol-Domain combination will have different local storage, which means, [http://example.com](http://example.com) and [https://example.com](https://example.com) will not share their local storage property.

Why use Local Storage?

LocalStorage is generally used to save a user's preferences, theme settings, login information, auth tokens. You can store any kind of information you want.

Using Local Storage

Let's start by creating a simple web page with a button and a heading 💻

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Local Storage</title>
</head>
<body>
    <button id="count-btn">Count</button>
    <h1 id="clicks-count">0</h1>

    <script src="main.js"></script>
</body>
</html>

We have given both button and h1 tags an id to access them in the JavaScript **file.

This is how our web page looks, we want to increase the number by 1 each time the Count button is pressed.

Let's create a JavaScript file main.js and the following code **to implement this behavior:

document.querySelector('#count-btn').addEventListener('click', (e) => {
    const countElement = document.querySelector('#clicks-count');
    const count = Number(countElement.innerText) + 1;
    document.querySelector('#count-btn').addEventListener('click', (e) => {
    const countElement = document.querySelector('#clicks-count');
    const count = Number(countElement.innerText) + 1;
    localStorage.setItem('count', count);
    countElement.innerText = count;
});
    countElement.innerText = count;
});

We added an event listener for the click action on the button, whenever the button is clicked we access the value of h1, increment it by 1, and then update the h1 again.

Now, the page is behaving the way we want:

But there's a problem! Try refreshing the page and the count resets to 0.

Now we will use local storage to save this count so that we start from the same number we reached before refreshing or closing the browser.

Setting count in LocalStorage

Use the localStorage.setItem(key, value); in the event listener we created before:

document.querySelector('#count-btn').addEventListener('click', (e) => {
    const countElement = document.querySelector('#clicks-count');
    const count = Number(countElement.innerText) + 1;
    localStorage.setItem('count', count);
    countElement.innerText = count;
});

This will set a key count in the local storage and update it each time we press the button.

You can check it by opening the inspect element (Control+Shift+I on windows, Command + Shift + I on mac) and going into the application tab, and selecting Local Storage from the left menu:

Try refreshing the page again, it still starts from 0, that's because we have saved the value but we also need to set the previous value of count to the initial value when the page loads:

We will use the localStorage.getItem(key); to access a value of a key from localStorage.

Edit the main.js to use getItem function:

document.querySelector('#clicks-count').innerText = localStorage.getItem('count') ?? 0;

document.querySelector('#count-btn').addEventListener('click', (e) => {
    const countElement = document.querySelector('#clicks-count');
    const count = Number(countElement.innerText) + 1;
    localStorage.setItem('count', count);
    countElement.innerText = count;
});

The first line checks if a key 'count' exists in the local storage, if true it sets the heading to that number otherwise it set it to 0.

You could also have used an if condition for this but here we are using the more succinct ?? operator.

?? - The nullish coalescing operator (??) is a logical operator that returns its right-hand side operand when its left-hand side operand is null or undefined and otherwise returns its left-hand side operand.

And with this, we are done with our web page. Try refreshing or closing the browser and the count will still start from the last value.

Local Storage Methods Summary

• localStorage.setItem(key, value);

  Adds a key and value pair to local storage.

• **const cat = localStorage.getItem(key);**

  Gets a value for a key from the local storage.

• **localStorage.removeItem(key);**

  Deletes a key-value pair from the local storage.

• localStorage.clear();

  Deletes all key-value pairs from the local storage.

Conclusion

Now that you know how to use localStorage, try using it in your next project.

Here are a few project ideas for beginners to try out localStorage:

• Todo App
• Notes App
• Shopping List App

Note: Just because you can store any information in local storage doesn't mean you should. Never store sensitive data in local storage as any third party js code would be able to access it, increasing the security risk. (Thanks to @apoorvtyagi for questioning about the security of local storage)

I hope you found this article helpful! Thanks for reading.

Note: This is an article I wrote in 2021 and never published for three years. Some part of it might not work today but it still is helpful for anyone looking to go from knowing just the fundamentals of Python to building an end-to-end API in fastAPI.

Introduction

Hey Everyone! In this article, we will go through all the features of FastAPI needed to build an API for a Todo App. We will perform CRUD(create-read-update-delete) operations, add a database and authentication to our API.

We will start by understanding a few prerequisites for the FastAPI app, then we will go through a hello world app for FastAPI and learn its fundamentals, and then finally end by building a todo app with FastAPI.

The only prerequisite to this article is to have experience coding in Python.

The best way to go through the article is to open your code editor and code along with me. (Don't copy-paste)

So let us start!

What is FastAPI?

FastAPI is a modern, fast (high-performance) web framework for building APIs with Python 3.6+.

But wait, what is an API? (feel free to skip if you already know)

API stands for Application programming interface, it is a set of functions that performs a task and can be used by other developers to build their own software without caring about the implementation detail of the API.

One of the most famous analogies of an API is to think of an API as a waiter in a restaurant, when you go to a restaurant a waiter takes your order, delivers it to the kitchen, and then delivers the food.

You may watch this 3 minutes video that explains API in a precise way:

Why use FastAPI to build APIs?

The answer is simple: FastAPI is awesome!

• It is Fast: FastAPI is a high-performance web framework, on par with NodeJS and Go.

• It is Fast to code: Increase the speed to develop features by about 200% to 300%.

• It is Intuitive: Great editor support. Completion everywhere. Less time debugging.

• It is Easy: Easy to use and learn.

• It is Short: Minimize code duplication.

• It is Robust: Get production-ready code. With automatic interactive documentation.

• It is Standard-bases: Based on (and fully compatible with) the open standards for APIs.

FastAPI uses some of the best libraries available in the Python community to achieve the above features:

1. Pydantic: For data validation, serialization, and documentation.

2. Starlette: Starlette is a lightweight ASGI web framework/toolkit.

3. Uvicorn: Uvicorn is a lightning-fast ASGI server.

Don't worry if don't understand a few terms here, we will go through all this as needed.

Python Type Hints

Before starting development with FastAPI, we also need to make sure we are comfortable using types in Python.

(Feel free to skip this section if you are already familiar with type hints in Python)

Type hints are a special syntax that allows declaring the type of a variable.

Editors and tools use these types of hints to provide better support like auto-completion and error checks.

FastAPI is based on these types hints and uses them to provide:

• Data validation

• Serialization

• Documentation support.

Using type hints

Type hints are declared using a : after the variable name, see the following example:

def add(num1: int, num2: int):
    return num1 + num2

You can also use any standard python types: int, float, str, bool, bytes

Using the typing module

For data structures that can contain other values, like dict, list, set, tuple you can use the standard python module typing. These types are called generic types.

Here's an example:

from typing import List

def process_items(items: List[str]):
    for item in items:
        print(item)

List[str]: The type of the values that a list can contain is given in square brackets []. It is called the type parameter.

That means: "the variable items is a list, and each of the items in this list is an str".

Tuple and Set: For declaring the type of tuples and sets:

from typing import Set, Tuple

def process_items(items_t: Tuple[int, int, str], items_s: Set[bytes]):
    return items_t, items_s

This means:

• The variable items_t is a tuple with 3 items, an int, another int, and str.

  If you want to declare a tuple with variable length and homogeneous type, you can use literal ellipsis: Tuple[int, ...].

  If you want to declare a plain tuple: Tuple[Any, ...].

• The variable items_s is a set, and each of its items is of type bytes.

Dicts:

To define a dict, you pass 2 type parameters, separated by commas.

from typing import Dict

def process_items(prices: Dict[str, float]):
    for item_name, item_price in prices.items():
        print(item_name)
        print(item_price)

This means:

• The variable prices is a dict:

  • The keys of this dict are of type str.

  • The values of this dict are of type float.

Optional

You can also use Optional to declare that a variable has a type, like str, but that it is "optional", which means that it could also be None.

from typing import Optional

def say_hi(name: Optional[str] = None):
    if name is not None:
        print(f"Hey {name}!")
    else:
        print("Hello, World!")

Using Optional[str] instead of just str will let the editor help you detecting errors where you could be assuming that a value is always str when it could actually be None too.

Union

Union types allow combining types: Union[int, str] means either int or str.

Literal

Literal types can be used to indicate that the corresponding variable can have one of the provided literal value: MODE = Literal['r', 'rb', 'w', 'wb'].

Using classes as type hints

You can also declare a class as the type of a variable.

class Person:
    def __init__(self, name: str):
        self.name = name

def get_person_name(a_person: Person):
    return a_person.name

Note 📝

Since Python 3.9 you can use built-in collection types such as list and dict as generic types instead of importing the corresponding types (ex. List and Dict) from typing module.

def greet_all(names: list[str]) -> None:
    for name in names:
        print("Hello", name)
# The -> indicate return type although it can be inferred automatically

Tip 💡

Install the mypy library to get error checks when using type hints.

Install $ python3 -m pip install mypy.

Setup in VS Code: Open the command palette (Ctrl+Shift+P) and select the Python: Select Linter command and choose mypy. Also, make sure you have the pylance and the python extension installed in your vs code for the best experience.

Hello World with FastAPI

Now, we will create a Hello World app in FastAPI and then discuss a few fundamental features.

Let's start by creating a new project folder:

$ mkdir hello
$ cd hello
$ touch __init__.py # this will create an __init__.py file to make this folder a package

Now, create a virtual environment and activate it:

$ python3 -m venv env
$ source env/bin/activate

Time to Install FastAPI:

$ pip install fastapi[all]

Hello, World!

Now, create a file [main.py](http://main.py) in the current directory and type the following into it, don't worry we will go over each line.

# Snippet 1

from fastapi import FastAPI

app = FastAPI()

@app.get("/")
async def hello():
    return {"message": "Hello, World!"}

• from fastapi import FastAPI

  We are importing the FastAPI class from the fastapi module, it will provide all the functionality for our API, and make our app a FastAPI app.

• app = FastAPI()

  This creates an instance of the FastAPI class. We will also refer to this instance when running our API using a server. Also, you can name it anything instead of the app.

• @app.get("/")

  This line creates a path operation using a decorator.

  A Path refers to the last part of the URL starting from the first /.

  So, in a URL like: [https://example.com/items/foo](https://example.com/items/foo), the path would be /items/foo.

  An operation refers to one of the HTTP methods:

  • POST: to create data

  • GET: to read data

  • PUT: to update data

  • DELETE: to delete data

  • and more...

You can communicate to each path using one (or more) of these "methods" in the HTTP protocol.

So, the @app.get("/") decorator (a "decorator" takes the function below and does something with it) tells FastAPI that the function right below is in charge of handling requests that go to:

• The path /.

• Using a get operation.

You can also use the other operations:

• @app.post()

• @app.put()

• @app.delete()

• The hello() function

  This is a path operation function, it is called by FastAPI whenever it receives a request to the URL / using a GET operation. You can use a normal function or an async function.

  This function is returning a python dictionary which will be automatically converted to a JSON object by FastAPI.

  • JSON stands for JavaScript Object Notation.

  • It is a lightweight format for storing and transporting data.

  • JSON is often used when data is sent from a server to a web page.

  • JSON is very similar to a Python dictionary, so if you haven't ever used JSON then don't worry just think of it as a slightly different representation of a python dictionary.

You can also return other data types like list, str, int, etc.

Running a FastAPI app

Let's run the above hello world app, open your terminal and type the following command:

$ uvicorn main:app --reload

This will start our app on [localhost:8000](http://localhost:8000).

main refers to our Python module and app refers to the FastAPI instance we created.

--reload is used to restart the server after code changes. Only use for development.

The Autogenerated Docs

FastAPI provides automatic documentation, go to [localhost:8000/docs](http://localhost:8000/docs) and you will be able to see all the path operations and try them out.

FastAPI Fundamentals

Now, we will create more path operations in the same hello world app and understand a few important concepts in FastAPI like path parameters, query parameters, request body, using Pydantic models, response models, and dependencies.

We will learn about Routers, Databases, and Security while creating the Todo API in the next Section.

Path Parameters

Path parameters are the variable parts of the URL path.

In [https://twitter.com/yuvraajsj18](https://twitter.com/yuvraajsj18), yuvraajsj18 is a path parameter. You can insert any other value: https://twitter.com/{username}.

Let's create our own path with a path parameter:

# Snippet 2

# ... code from snippet 1

items = [
    {
        "item_id": 0,
        "name": "Item 1",
        "description": "Description for item 1",
        "price": 50.25,
    },
    {
        "item_id": 1,
        "name": "Item 2",
        "description": "Description for item 2",
        "price": 20.00,
        "tax": 2.0,
    },
    {
        "item_id": 2,
        "name": "Item 3",
        "description": "Description for item 3",
        "price": 30.00,
        "tax": 1.5,
    },
]

**@app.get("/items/{item_id}")
async def get_item(item_id: int):
    if item_id < len(items):
        return items[item_id]

    return {"Error": f"item_id {item_id} not found."}**

First we have created a items list to mock a database.

Then we are creating a get path operation at items/{item_id}.

{item_id} is the placeholder for our path parameter.

Whatever is passed into the URL will be automatically converted, validated, and saved into the int variable item_id declared in the function signature: get_item(item_id: int).

We can then use it in the definition. Here we retrieve the item and return the corresponding item dictionary if it exists otherwise an error message.

Try running this using the uvicorn server and going to the docs path.

Also, try passing a string parameter in the URL like [localhost:8000/items/foo](http://localhost:8000/items/foo) instead of int. You will get the following error:

{"detail":[{"loc":["path","item_id"],"msg":"value is not a valid integer","type":"type_error.integer"}]}

Query Parameters

Query Parameters are the key-value pairs that are provided after the ? and separated by & in the URL.

In the URL: https://www.google.com/search?q=cats, q=cats is a query parameter, it tells google what "query" is to search.

In FastAPI, When you declare other function parameters that are not part of the path parameters, they are automatically interpreted as "query" parameters.

Let's continue with our file written above and add a path operation that accepts query parameters.

# Snippet 3

# ... code from snippet 2

@app.get("/items/")
async def read_items(skip: int = 0, limit: int = 10):
    return items[skip : skip + limit]

Try going to the [localhost:8000/items/?skip=1&limit=3](http://localhost:8000/items/?skip=1&limit=3), you will get the items[1: 3] that is the following 2 items:

[{"item_id":1,"name":"Item 2","description":"Description for item 2","price":20.0,"tax":2.0},
{"item_id":2,"name":"Item 3","description":"Description for item 3","price":30.0,"tax":1.5}]

FastAPI receives these query parameters from the URL as string, convert to their declared type, int in this case and store them in the corresponding variable.

As query parameters are not a fixed part of a path, they can be optional and can have default values.

In the example above they have default values of skip=0 and limit=10. So, going to [localhost:8000/items](http://localhost:8000/items) is same as going to localhost:8000/items/?skip=0&limit=10.

You can also create optional query parameters using None, let's add a greet function to illustrate:

# Snippet 4
from typing import Optional
# ... code from snippet 3

@app.get("/greet")
async def greet(name: Optional[str]= None):
    if name:
        return {"message": f"Hello, {name}"}
    return {"message": "Hello, Anonymous!"}

FastAPI will know that name is optional because of the = None.

The Optional in Optional[str] is not used by FastAPI (FastAPI will only use the str part), but the Optional[str] will let your editor help you find errors in your code.

When you want to make a query parameter required, you can just not declare any default value.

Request Body

When you need to send data from a client (let's say, a browser) to your API, you send it as a request body.

A request body is data sent by the client to your API.

A response body is the data your API sends to the client.

Example: When you fill up a sign-up form on a platform, the data is sent to the back-end using the request body.

To declare a request body, we use Pydanticmodels.

Let's create a path operation that adds a new item to our list of items:

# Snippet 5

from pydantic import BaseModel

# ... code from snippet 4

class Item(BaseModel):
    name: str
    description: Optional[str] = None
    price: float
    tax: Optional[float] = None

# To receive request body, you should use one of: 
# POST (the more common), PUT, DELETE or PATCH.
@app.post("/items/")
async def create_item(item: Item):
    items.append({"item_id": len(items), **item.dict()})
    return items[-1]

The Item class is a pydantic model that defines the shape of an item. When an instance of Item is created, the attributes used will be validated, parsed, and stored in the instance.

FastAPI will receive the JSON object from the client, and provide us with the corresponding pydantic model instance, in this case, item.

We can define optional attributes by assigning None to them, as done for the description and tax.

We will then use the item instance to append a new item in the items list. **item.dict() is dictionary unpacking syntax in python. It will return the keyword-value pair. We are also adding an item_id and then finally returning the newly created item.

Try Creating an Item using the /docs path:

The new item is created but it will not persist once you restart your app, we will use databases in the Todo API section to make data persistent.

You can use body, path, query parameters together. FastAPI will recognize each of them and take the data from the correct place.

Adding additional information and validation to parameters

We can use Query, Path, Field, Body to provide an additional string or numeric validation for our parameters.

Let's add string validation to the greet function we have written before:

# Snippet 4 Modified
from fastapi import FastAPI, Query  # import Query
from typing import Optional

@app.get("/greet")
async def greet(
    **name: Optional[str] = Query(None, min_length=1, title="Name", alias="username")**
):
    if name:
        return {"message": f"Hello, {name}"}
    return {"message": "Hello, Anonymous!"}

The first value to Query provides the default value for the parameter. It is followed by other optional validations:

• min_length: int, specifies the minimum length of the parameter value.

• max_length: int, specifies the maximum length of the parameter value.

• regex: str, defines a regular expression that the parameter should match.

• alias: str, provides an alternate path for the operation, now greet can be called with /greet/?username=Yuvraj instead of /greet/?name=Yuvraj.

• title: str, provides title to the query in the docs, a metadata argument.

• description: str, describes the query in the docs, a metadata argument.

• deprecated: bool, specifies whether the parameter is deprecated or not in the docs, a metadata argument.

When you don't want to provide a default value, you can use Query(..., min_length=1). ... is a special python value called ellipsis.

Now, let's add some numeric validation to the read_items function written before:

# Snippet 2 Modified
from fastapi import FastAPI, **Path,** Query

@app.get("/items/{item_id}")
**async def get_item(item_id: int = Path(..., ge=0)):**
    if item_id < len(items):
        return items[item_id]

    return {"Error": f"item_id {item_id} not found."}

As a path parameter is always required, the first argument to Path should always be the... It can be followed by the metadata arguments or the numeric validation arguments:

• gt: greater than

• ge: greater than or equal

• lt: less than

• le: less than or equal

Both string and numeric validation and metadata arguments can be used with Query and Path.

There also exists Field to add similar validation to a Pydantic model field and Body to add validation when receiving a singular value from the client instead of a dictionary.

Here is the example of both Field and Body

# Snippet 5 modified

from fastapi import **Body**, FastAPI, **HTTPException** Path, Query
from pydantic import BaseModel, **Field**

class Item(BaseModel):
    name: str
    description: Optional[str] = Field(None, max_length=300)
    price: float = Field(..., gt=0, description="The price must be greater than zero")
    tax: Optional[float] = None

@app.post("/items/")
async def create_item(item: Item):
    items.append({"item_id": len(items), **item.dict()})
    return items[-1]

@app.put("/items/{item_id}")
async def update_item(item_id: int, add_price: float = Body(...)):
    if item_id >= len(items):
        **raise HTTPException(status_code=404, detail="Item not found")**

    item = Item(**items[item_id])
    item.price += add_price

    items[item_id] = item.dict()

    return items[item_id]

• update_item expect a single value for add_price.

Handling Exceptions

In the above code, in function update_item we saw the use of HTTPException.

There are many situations where you need to notify an error to a client that is using your API. We can do it with the help of HTTPException class.

HTTPException is a normal Python exception with additional data relevant for APIs.

Because it's a Python exception, you don't return it, you raise it.

This also means that if you are inside a utility function that you are calling inside of your path operation function, and you raise the HTTPException from inside of that utility function, it won't run the rest of the code in the path operation function, it will terminate that request right away and send the HTTP error from the HTTPException to the client.

Following are the status codes to specify the response types:

1. Informational responses (100–199)

2. Successful responses (200–299)

3. Redirects (300–399)

4. Client errors (400–499)

5. Server errors (500–599)

Response Model

We can declare the model used for the response with the response_model in any of the path operations.

Let's modify the create_item operation to include a response model:

@app.post("/items/", response_model=Item)
async def create_item(item: Item):
    items.append({"item_id": len(items), **item.dict()})
    return items[-1]

Now go to the /docs and see the response section of the create_item, you will see the Item model instead of the string that was shown before.

FastAPI will use this response_model to:

• Convert the output data to its type declaration.

• Validate the data.

• Add a JSON Schema for the response, in the OpenAPI path operation.

• Will be used by the automatic documentation systems.

• Will limit the output data to that of the model.

  This means now when you run this operation, the item_id won't be returned anymore. As it is not included in the Item model. You can create multiple models for the request (input) and response (output), we will see this in detail in the Todo API app.

Dependencies

Okay, one last topic before moving on to creating the Todo API with FastAPI and using databases, routers, and authentication.

Dependency injection means, that there is a way for our code to declare things that it requires to work and use: "dependencies".

FastAPI will take care of doing whatever is needed to provide your code with those needed dependencies.

This is very useful when you need to:

• Have shared logic (the same code logic again and again).

• Share database connections.

• Enforce security, authentication, role requirements, etc.

• And many other things...

All these, while minimizing code repetition.

Let's implement a dependency in our hello app:

Remember we created a read_items function that takes two parameters skip and limit. Now we will do the same for a users list.

users = [
    {
        "id": 0,
        "username": "user1",
        "hashed_password": "somecrypticpassword1",
    },
    {
        "id": 1,
        "username": "user2",
        "hashed_password": "somecrypticpassword2",
    },
    {
        "id": 2,
        "username": "user2",
        "hashed_password": "somecrypticpassword2",
    },
]

@app.get("/users/")
async def read_users(skip: int = 0, limit: int = 10):
    return users[skip : skip + limit]

Now we will use a dependency to factor out this common parameter.

async def common_parameters(skip: int = 0, limit: int = 10):
    return {"skip": skip, "limit": limit}

See, a dependency is just a function. Here it is taking the same parameter as the path functions.

Now we will use this in our path functions:

from fastapi import Body, **Depends**, FastAPI, HTTPException, Path, Query

@app.get("/items/")
async def read_items(commons: dict = Depends(common_parameters)):
    return items[commons["skip"] : commons["skip"] + commons["limit"]]

@app.get("/users/")
async def read_users(commons: dict = Depends(common_parameters)):
    return users[commons["skip"] : commons["skip"] + commons["limit"]]

Whenever a new request arrives, FastAPI will take care of:

• Calling your dependency ("dependable") function with the correct parameters.

• Get the result from your function.

• Assign that result to the parameter in your path operation function.

Try running the app using /docs. It still functions the same way as before.

We can also use classes as a dependency:

class CommonQueryParams:
    def __init__(self, skip: int = 0, limit: int = 10):
        self.skip = skip
        self.limit = limit

@app.get("/items/")
async def read_items(**commons: CommonQueryParams = Depends(CommonQueryParams)**):
    return items[commons.skip : commons.skip + commons.limit]

@app.get("/users/")
async def read_users(**commons: CommonQueryParams = Depends(CommonQueryParams)**):
    return items[commons.skip : commons.skip + commons.limit]

• Using classes allows your editor to provide code completion.

• You can use the following shortcut to avoid typing the type twice when using class dependency:

    commons: CommonQueryParams = Depends()
  

Few things to remember while using dependencies:

• You can create dependencies that have sub-dependencies. This means one dependency can be dependent on another.

• If one of your dependencies is declared multiple times for the same path operation, for example, multiple dependencies have a common sub-dependency, FastAPI will know to call that sub-dependency only once per request.

• When you don't need the return value of a dependency, you can add a dependencies list in the path operation decorator instead of declaring it in the path operation function.

    @app.get("/items/", dependencies=[Depends(dependency1), Depends(dependency2)])
    async def read_items():
        return items
  

• You can add dependencies to all the path operations by creating global dependencies on the app:

    app = FastAPI(dependencies=[Depends(dependency1), Depends(dependency2)])
  

• You can perform extra steps after finishing the dependency using yield instead of return.

  When the Python yield statement is encountered, the program suspends function execution, saves its state, and returns the yielded value to the caller. Whereas return stops function execution completely.

  You could use this to create a database session and close it after finishing.

  Only the code before and including the yield statement is executed before sending a response:

    async def get_db():
        db = DBSession()
        try:
            yield db
        finally:
            db.close()
  

  Note: you cannot raise an HTTPException after the yield statement as the response has already been sent to the user.

We will now put all of this together and use more features like databases, auth, and routing to create a Todo API.

Building the Todo API

Let's start by creating a new project:

$ mkdir todo
$ cd todo
$ touch __init__.py

Then, create and activate a virtual environment:

$ python3 -m venv env
$ source env/bin/activate

Install FastAPI

$ pip install fastapi[all]

Now, before going any further, let's define the requirements of our API.

Requirements:

1. User registration.

2. User authentication.

3. Create a new to-do item.

4. Delete an existing to-do item.

5. Mark a to-do item as done/not done.

6. View all to-do items.

Create a directory structure as follows:

todo ├── app │ ├── crud

│ ├── db │ ├── models │ ├── routers │ └── schemas └── env

Authentication

We will use tools provided by FastAPI to build our authentication system using OAuth2.

OAuth2 is a specification that defines several ways to handle authentication and authorization.

Let's see what are the tools provided by FastAPI for authentication(you don't have to code this yet, we will utilize these tools in Todo API later):

from fastapi import Depends, FastAPI
from fastapi. security import OAuth2PasswordBearer

app = FastAPI()

oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token")

@app.get("/items/")
async def read_items(token: str = Depends(oauth2_scheme)):
    return {"token": token}

When we run this and try out the /items operation we will be required to provide the username and password.

This is the password flow defined in OAuth2. In this flow, the API itself is handling the authentication.

This is how everything works in this flow:

• The user types the username and password in the frontend and hits Enter.

• The frontend (running in the user's browser) sends that username and password to a specific URL in our API (declared with tokenUrl="token").

• The API checks that username and password, and responds with a "token" (we haven't implemented any of this yet).

  • A "token" is just a string with some content that we can use later to verify this user.

  • Normally, a token is set to expire after some time.

    • So, the user will have to log in again at some point later.

    • And if the token is stolen, the risk is less. It is not like a permanent key that will work forever (in most cases).

• The front end stores that token temporarily somewhere.

• The user clicks in the frontend to go to another section of the frontend web app.

• The front end needs to fetch some more data from the API.

  • But it needs authentication for that specific endpoint.

  • So, to authenticate with our API, it sends a header Authorization with a value of Bearer plus the token.

    (a header is just another part of the request sent by the client. Like the body, as we have seen before).

  • If the token contains foobar, the content of the Authorization header would be Bearer foobar.

FastAPI's OAuth2PasswordBearer

oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token")

We are creating an instance oauth2_scheme of the class OAuth2PasswordBearer by passing a URL that will receive the username and password. This parameter doesn't create the path operation, we will do it ourselves later.

The instance oauth2_scheme is a callable, so it can be used with Depends.

Usingoauth2_scheme

@app.get("/items")
async def read_items(**token: str = Depends(oauth2_scheme)**):
    return {"token": token}

This dependency will go and look in the request for that Authorization header, check if the value is Bearer plus some token, and return the token as str that will be assigned to token the path operation function.

If it doesn't see an Authorization header, or the value doesn't have a Bearer token, it will respond with a 401 status code error (UNAUTHORIZED) directly.

You don't even have to check if the token exists to return an error. You can be sure that if your function is executed, it will have str in that token.

UsingUser Model

Let's make the auth system useful by creating a User model. (Again, you don't have to code this right now, we will be creating a better version for Todo API later).

from pydantic import BaseModel

class User(BaseModel):
    username: str
    email: Optional[str] = None
    full_name: Optional[str] = None

Let's create a dependency get_current_user:

def fake_decode_token(token: str):
    return User(
            username=token + "fakedecoded", email="john@example.com", full_name="John Doe"
    )

**async def get_current_user(token: str = Depends(oauth2_scheme)):**
    user = fake_decode_token(token)
    return user

get_current_user is a dependency that has oauth2_scheme as a sub-dependency. We are also using a fake utility function for now to decode tokens.

Inject the current user

Now, we will use get_current_user dependency in a path operation:

@app.get("/users/me")
async def read_users_me(current_user: User = Depends(get_current_user)):
    return current_user

Getting theusername and password from client:

OAuth2 specifies that when using the "password flow" the client/user must send a username and password fields as form data.

The fields should also be named like this, email and password won't work. Although we can show the fields to the final users in the frontend as we wish.

Also, database models can use other names too.

UsingOAuth2PasswordRequestForm

from fastapi.security import OAuth2PasswordBearer, **OAuth2PasswordRequestForm**

fake_users_db = {
    "johndoe": {
        "username": "johndoe",
        "full_name": "John Doe",
        "email": "johndoe@example.com",
        "hashed_password": "fakehashedsecret",
    },
    "alice": {
        "username": "alice",
        "full_name": "Alice Wonderson",
        "email": "alice@example.com",
        "hashed_password": "fakehashedsecret2",
    },
}

class UserInDB(User):  # User is same as defined before
    hashed_password: str

def fake_hash_password(password: str):
    return "fakehashed" + password

**@app.post("/token")
async def login(form_data: OAuth2PasswordRequestForm = Depends()):**
    user_dict = fake_user_db.get(form_data.username)

    if not user_dict:
        raise HTTPException(status_code=400, detail="Incorrect username or password")

    user = UserInDB(**user_dict)
    hashed_password = fake_hash_password(form_data.password)

    if not hashed_password == user.hashed_password:
        raise HTTPException(status_code=400, detail="Incorrect username or password")

    return {"access_token": user.username, "token_type": "bearer"}

The OAuth2PasswordRequestForm is a class dependency that provides username and password received as form data.

(Form data in fastAPI is received using Form. It's similar to Body or Query. But instead of application/json encoding it uses application/x-www-form-urlencoded encoding. The <form></form> tag in HTML uses this encoding. Although you don't have to worry about this much as FastAPI will provide you with data directly just like it did with the body).

Then, we are getting the user data from the fake database using the username field. If no such user is found we are raising an error.

Now, we put the data in the pydantic model UserInDB and check if the hash of the password field matches the user's hashed password. If the passwords don't match, we raise an error.

Password Hashing

"Hashing" means: converting some content (a password in this case) into a sequence of bytes (just a string) that looks like gibberish.

Whenever you pass the same content (the same password) you get the same gibberish.

But you cannot convert from the gibberish back to the password.

Why use password hashing?

If your database is stolen, the thief won't have your users' plaintext passwords, only the hashes.

Returning the token:

The response of the token endpoint must be a JSON object.

It should have a token_type. In our case, as we are using "Bearer" tokens, the token type should be "bearer".

And it should have an access_token, with a string containing our access token.

In this example, we are returning the username as a token that is not secure, we will implement a secure token in the Todo API.

Update the dependencies

We will handle an exception in the previously written get_current_user dependency:

from fastapi import Depends, FastAPI, HTTPException, **status**

def get_user(db, username: str):
    if username in db:
        user_dict = db[username]
        return UserInDB(**user_dict)

def fake_decode_token(token: str):
    user = get_user(fake_users_db, token)
    return user

**async def get_current_user(token: str = Depends(oauth2_scheme)):**
    user = fake_decode_token(token)
    if not user:
        raise HTTPException(
            status_code=status.HTTP_401_UNAUTHORIZED,
            detail="Invalid authentication credentials",
            headers={"WWW-Authenticate": "Bearer"},
        )
    return user

The additional header WWW-Authenticate with value Bearer we are returning here is also part of the spec.

Any HTTP (error) status code 401 "UNAUTHORIZED" is supposed to also return a WWW-Authenticate header.

This is how you implement an insecure authentication, now we will implement a secure version for our Todo API, get ready to code.

Adding Authentication Todo API

We will use the password flow with hashing for our Todo API and use JWT token instead of insecure usernames.

(You can now code along)

JWT

JWT means "JSON Web Tokens". It's a standard to codify a JSON object in a long dense string without spaces.

Example: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c

It is not encrypted, so, anyone could recover the information from the contents.

But it's signed. So, when you receive a token that you emitted, you can verify that you actually emitted it.

That way, you can create a token with an expiration of, let's say, 1 week. And then when the user comes back the next day with the token, you know that the user is still logged in to your system.

After a week, the token will be expired and the user will not be authorized and will have to sign in again to get a new token. And if the user (or a third party) tried to modify the token to change the expiration, you would be able to discover it, because the signatures would not match.

Installpython-jose to generate and verify the JWT tokens in Python:

$ pip install python-jose[cryptography]

Password Hashing

Install PassLib to handle password hashes.

It supports many secure hashing algorithms and utilities to work with them. The recommended algorithm is "Bcrypt".

$ pip install passlib[bcrypt]

Let's create a file security.py inside the app directory:

from passlib.context import CryptContext

pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")

def verify_password(plain_password: str, hashed_password: str):
    return pwd_context.verify(plain_password, hashed_password)

def get_password_hash(password: str):
    return pwd_context.hash(password)

We are creating a PassLib "context" that we will use to hash and verify passwords.

Handle JWT tokens

As told before, we are required to send an access token and token type as a response from the /token endpoint. That's why create a file [token.py](http://token.py) inside the schema folder to store the following Pydantic models. Token will be later used in the /token endpoint and TokenData will be used to store the decoded token payload.

from typing import Optional
from pydantic import BaseModel

class Token(BaseModel):
    access_token: str
    token_type: str

class TokenData(BaseModel):
    username: Optional[str] = None

Note: Add the following line to __init__.py inside schemas/ to simplify imports: from .token import Token, TokenData

Now, create a random secret key to sign the JWT tokens, execute the following command in your terminal:

$ openssl rand -hex 32

You will get a random key in the output like this:

939aae726cc09e3338ea836b90b0b0a4dffce9e8ed069ac343754c236244e3d2

Save this key in an environment variable(don't use this one). You can use a .env file to store it so that you don't have to manually set them every time you start a new shell.

create .env file in app folder:

SECRET_JWT_KEY=939aae726cc09e3338ea836b90b0b0a4dffce9e8ed069ac343754c236244e3d2

Install the python-dotenv package to import variables from the .env file into the environment: pip install python-dotenv.

Note: if you are using git, make sure to add .env in the .gitignore file as it contains sensitive information.

Let's add functions to handle tokens and authentication in the security.py file:

import os
from datetime import datetime, timedelta
from typing import Optional

from dotenv import load_dotenv
from fastapi import HTTPException, status
from jose import JWTError, jwt

from app import schemas

load_dotenv()  # load the variables from .env

SECRET_JWT_TOKEN = str(os.environ["SECRET_JWT_KEY"])
ALGORITHM = "HS256"

def create_access_token(data: dict, expires_delta: Optional[timedelta] = None):
    to_encode = data.copy()
    if expires_delta:
        expire = datetime.utcnow() + expires_delta
    else:
        expire = datetime.utcnow() + timedelta(minutes=15)

    to_encode.update({"exp": expire})
    encoded_jwt = jwt.encode(to_encode, SECRET_JWT_TOKEN, algorithm=ALGORITHM)

    return encoded_jwt

def decode_access_token(token: str):
    credentials_exception = HTTPException(
        status_code=status.HTTP_401_UNAUTHORIZED,
        detail="Could not validate credentials",
        headers={"WWW-Authenticate": "Bearers"},
    )
    try:
        payload = jwt.decode(token, SECRET_JWT_TOKEN, algorithms=[ALGORITHM])
        username: str = payload.get("sub")
        if username is None:
            raise credentials_exception
        token_data = schemas.TokenData(username=username)
    except JWTError:
        raise credentials_exception

    return token_data.username

Here, we are setting the variables for our secret key, algorithm, and expiration for the JWT token.

We are also defining a Pydantic model to use as a response model in the /token endpoint. TokenData model will store the data from the decoded JWT.

Then we finally created a function to generate an access token.

Note: We are using the crud module we haven't created yet, we will implement the CRUD functionality in the database section.

Creating auth dependencies

Create a file [dependency.py](http://dependency.py) in the app directory:

from fastapi import Depends, HTTPException, status
from fastapi.security import OAuth2PasswordBearer, oauth2
from sqlalchemy.orm import Session

from app import crud
from app.security import decode_access_token

oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token")

async def get_current_user(db: Session = Depends(get_db), token: str = Depends(oauth2_scheme)):
    username = decode_access_token(token)
    user = crud.get_user_by_username(db, username)
    if not user:
        raise HTTPException(
            status_code=status.HTTP_404_NOT_FOUND,
            detail="Could not validate credentials",
            headers={"WWW-Authenticate": "Bearer"},
        )

    return user

Token Route

Finally, let's create the endpoint for /token in the [main.py](http://main.py) inside app/:

(The get_db dependency will be created in the database section)

from datetime import timedelta

from fastapi import Depends, FastAPI, HTTPException, status
from fastapi.security import OAuth2PasswordRequestForm
from sqlalchemy.orm import Session

from app import crud, schemas, security
from app.dependency import get_db

app = FastAPI()

@app.get("/")
async def root():
    return {"app": "Todo API"}

ACCESS_TOKEN_EXPIRE_MINUTES = 30

@app.post("/token", response_model=schemas.Token)
async def login(
    db: Session = Depends(get_db), form_data: OAuth2PasswordRequestForm = Depends()
):
    user = crud.authenticate_user(db, form_data.username, form_data.password)

    if not user:
        raise HTTPException(
            status_code=status.HTTP_401_UNAUTHORIZED,
            detail="Invalid username or password",
            headers={"WWW-Authenticate": "Bearer"},
        )

    access_token_expires = timedelta(minutes=ACCESS_TOKEN_EXPIRE_MINUTES)
    access_token = security.create_access_token(
        data={"sub": user.username}, expires_delta=access_token_expires
    )
    return {"access_token": access_token, "token_type": "bearer"}

Here authenticate_user will be implemented in the next section of CRUD operations. It just checks if the user exists and has the correct password or not and returns a Boolean value.

JWT "subject" sub: The JWT specification says that there's a key sub, with the subject of the token. It's optional to use it, but that's where you would put the user's identification, so we are using it here.

The "sub" key should have a unique identifier across the entire application, and it should be a string.

Adding endpoints in the User route

Create a file [users.py](http://users.py) in the routers folder:

(User schema will be created in the database section)

from app import schemas
from fastapi import APIRouter
from fastapi.param_functions import Depends
from app.dependency import get_current_user

router = APIRouter(prefix="/users", tags=["users"])

@router.get("/me", response_model=schemas.User)
async def read_users_me(current_user: schemas.users.User = Depends(get_current_user)):
    return current_user

To organize different path operations, we can use the APIRouter. It's similar to the FastAPI class and supports all the same options like dependencies, tags.

The prefix will be used for all the paths, so /me/ will become /users/me/.

We can also add a list of dependencies that will be added to all the path operations in the router and will be executed/solved for each request made to them.

Let's add this router to the [main.py](http://main.py) file:

from app.routers import users

app = FastAPI()

app.include_router(users.router)

Database

We have added the code authentication, but we need a database to store the users and perform operations on it before we can run it.

We will use the SQLite database for this article as Python has integrated support for it, although FastAPI doesn't require you to use a SQL (relational) database. You can use any relational or no SQL database with FastAPI.

(Later, for your production application, you might want to use a database server like PostgreSQL.)

ORMs

ORM stands for object-relational mapping, an ORM has tools to convert between objects in code and database tables(relations).

With an ORM, we can represent a table in a SQL database using a class. Each attribute of the class represents a column, with a name and a type.

Each instance of the class will represent a row in the table.

• SQL table —> ORM class

• SQL columns —> ORM class attribute

• SQL rows —> ORM class instances

In this article, we will use SQLAlchemy ORM.

Install SQLAlchemy by running:

$ pip install SQLAlchemy

Configuring SQLAlchemy

Let's start by configuring SQLAlchemy, create a file [database.py](http://database.py) inside the db folder.

from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker

SQLALCHEMY_DATABASE_URL = "sqlite:///./app/todo_app.db"

engine = create_engine(
    SQLALCHEMY_DATABASE_URL, connect_args={"check_same_thread": False}
)

SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)

Base = declarative_base()

We first imported the SQLAlchemy tools and then created a database URL to connect to an SQLite database.

If you want to use a PostgreSQL database, change the URL to:

SQLALCHEMY_DATABASE_URL = "postgresql://user:password@postgresserver/db"

SQLAlchemyengine

Then we created an engine that is the starting point for any SQLAlchemy application, we will use it later.

Note:

The argument:

connect_args={"check_same_thread": False}

is needed only for SQLite. It's not needed for other databases.

SessionLocal class

Each instance of the SessionLocal class will be a database session. The class itself is not a database session yet.

But once we create an instance of the SessionLocal class, this instance will be the actual database session.

Base class

The function declarative_base() returns a class that is used to create each of the database models or classes.

Creating the database models for User

The classes used to interact with the databases are called models.

This is different than the previously mentioned Pydantic models which are used for data validation. To avoid confusion we will keep Pydantic models in the schemas folder and database models in the models' folder.

Let's create a file [users.py](http://users.py) inside the models' directory.

from app.db.database import Base
from sqlalchemy import Column, Integer, String

class User(Base):
    __tablename__ = "users"
    id = Column(Integer, primary_key=True, index=True)
    username = Column(String, unique=True, nullable=False, index=True)
    hashed_password = Column(String, nullable=False)

We are using the Base class from [database.py](http://database.py) to create the User model.

The __tablename__ attribute tells SQLAlchemy the name of the table to use in the database for each of these models.

Creating model attributes/columns

Then we created the model(class) attributes: id, username, hashed_password.

Each attribute represents a column in the users' table, and the value is given using the Column from SQLAlchemy.

The type of attribute is the first argument in the Column.

Note: Add the following line to __init__.py inside models/ to simplify imports: from .users import User

Creating the Pydantic models for User

Create a file [users.py](http://users.py) inside the schemas directory:

from pydantic import BaseModel

class UserBase(BaseModel):
    username: str

class UserCreate(UserBase):
    password: str

class User(UserBase):
    id: int

    class Config:
        orm_mode = True

We have created three pydantic models, for the creation (UserCreate), reading(User), and one base(UserBase) that has common attributes.

Password is only required when creating a user and should not be sent in a response.

The reading model User contains an id attribute as we will already know its ID after creation from the database.

The Config class is used to provide configurations to Pydantic. Pydantic's orm_mode will tell the Pydantic model to read the data even if it is not a dict, but an ORM model (or any other arbitrary object with attributes).

This way, instead of only trying to get the id value from a dict, as in:

id = data["id"]

it will also try to get it from an attribute, as in:

id = data.id

And with this, the Pydantic model is compatible with ORMs, and you can just declare it in the response_model argument in your path operations.

Note: Add the following line to __init__.py inside schemas/ to simplify imports: from .users import User, UserCreate

Creating CRUD for User

Create a file [users.py](http://users.py) inside the crud folder to define the functions to interact with the database's users' table.

CRUD: Create, Read, Update, and Delete.

For users, we will only implement create and read. Later, for todos, we will implement all four.

Read Users

from app import models, schemas
from app.security import get_password_hash, verify_password
from sqlalchemy.orm import Session

def get_user(db: Session, user_id: int):
    return db.query(models.User).filter(models.User.id == user_id).first()

def get_user_by_username(db: Session, username: str):
    return db.query(models.User).filter(models.User.username == username).first()

def get_users(db: Session, skip: int = 0, limit: int = 100):
    return db.query(models.User).offset(skip).limit(limit).all()

We imported Session from sqlalchemy.orm, this will allow us to declare the type of the db parameters and have better type checks and completion in your functions.

Create Users

Add the following function to the same file:

  def create_user(db: Session, user: schemas.UserCreate):
    hashed_password = get_password_hash(user.password)
    db_user = models.users.User(username=user.username, hashed_password=hashed_password)
    db.add(db_user)
    db.commit()
    db.refresh(db_user)
    return db_user

• Create an SQLAlchemy model instance with your data.

• add that instance object to your database session.

• commit the changes to the database (so that they are saved).

• refresh your instance (so that it contains any new data from the database, like the generated ID).

Authenticate Users

Also, adds the following function:

def authenticate_user(db: Session, username: str, password: str):
    user = get_user_by_username(db, username)
    if not user:
        return False
    if not verify_password(password, user.hashed_password):
        return False
    return user

Note: Add the following line to __init__.py inside crud/ to simplify imports: from .users import get_user, get_user_by_username, get_users, create_user, authenticate_user.

Initializing the database and setting migrations with Alembic

We have written then needed SQLAlchemy code for the users' table, but we need to convert it into a real database, for this, we use a migration tool.

A "migration" is the set of steps needed whenever we change the structure of our SQLAlchemy models, add a new attribute, etc. to replicate those changes in the database, add a new column, a new table, etc.

We will use Alembic for this purpose.

We will cover the necessary parts of alembic in this article. You can learn more in the alembic docs.

Install alembic

$ pip install alembic

Create Migrations Environment

Run the following command inside the todo directory:

$ alembic init alembic

This will create a migration environment for us.

A migration environment is a directory of scripts:

todo/
    alembic/
        env.py
        README
        script.py.mako
        versions/

• alembic/: home of the migration environment.

• env.py: a Python script that is run whenever the alembic migration tool is invoked. It configures and generates an SQLAlchemy engine.

• script.py.mako: a Mako template file to generate new migration scripts inside versions.

• versions/: This directory holds the individual version scripts.

Configurealembic.ini

Alembic created a file alembic.ini in the todo folder. This is a file that the alembic script looks for when invoked.

Edit the following line:

sqlalchemy.url = driver://user:pass@localhost/dbname

to:

sqlalchemy.url = sqlite:///./app/todo_app.db

Creating migrations scripts

We can create migration scripts either manually or automatically, here we will use the automatic approach and manually edit the scripts if required.

Alembic can view the status of the database and compare it against the table metadata in the application, generating the “obvious” migrations based on a comparison.

Create a file base.py to contain all the models for alembic to see:

# Import all the models, so that Base has them before being
# imported by Alembic
from app.db.database import Base
from app.models.users import User

Open the [env.py](http://env.py) file inside the alembic folder and make the following changes to target_metadata = None:

from app.db.base import Base

target_metadata = Base.metadata

Also, make sure the run_migrations_online() function is same as follows:

def run_migrations_online():
    connectable = engine_from_config(
        config.get_section(config.config_ini_section),
        prefix="sqlalchemy.",
        poolclass=pool.NullPool,
    )

    with connectable.connect() as connection:
        context.configure(
            connection=connection, target_metadata=target_metadata
        )

        with context.begin_transaction():
            context.run_migrations()

Now, run the alembic revision command with the --autogenerate option to create the script.

$ alembic revision --autogenerate -m "Add users table"

• The -m flag adds a message to the script.

A new script is created inside the versions/ folder, we can alter it as needed but it's not required in this case.

Running the migration script

Running the script will reflect the newly made changes in the actual database.

Run the following command to get to the "the most recent" version of database - head

$ alembic upgrade head

Yay! The database is initialized and we have our table live in the database.

Interact with the database directly

Before interacting with the database through FastAPI, let's do it directly.

Download the SQLiteBrowser and open the todo_app.db file with it:

$ sqlitebrowser app/todo_app.db # Or use the GUI in windows

Using FastAPI to interact with the database

Creating database dependency

Open the [dependency.py](http://dependency.py) in the app folder and create the following dependency:

from app.database import SessionLocal

def get_db():
    db = SessionLocal()
    try:
        yield db
    finally:
        db.close()

We are using the SessionLocal class created in the [databases.py](http://databases.py) file.

We need to have an independent database session/connection (SessionLocal) per request, use the same session through all the requests, and then close it after the request is finished.

And then a new session will be created for the next request.

Creating Path Operations

Now, we will use the dependency created above in the path operations function:

Add the following functions in the [users.py](http://users.py) file inside routers, see how we are using get_current_user dependency to make authentication required for an operation.

from typing import List
from app import crud, schemas
from app.dependency import get_current_user, get_db
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy.orm import Session

router = APIRouter(prefix="/users", tags=["users"])

@router.post("/", response_model=schemas.User)
async def create_user(user: schemas.UserCreate, db: Session = Depends(get_db)):
    db_user = crud.get_user_by_username(db, username=user.username)
    if db_user:
        raise HTTPException(
            status_code=status.HTTP_400_BAD_REQUEST, detail="Username already exists"
        )
    return crud.create_user(db, user)

@router.get(
    "/", response_model=List[schemas.User], dependencies=[Depends(get_current_user)]
)
async def read_users(skip: int = 0, limit: int = 100, db: Session = Depends(get_db)):
    users = crud.get_users(db, skip=skip, limit=limit)
    return users

@router.get("/me", response_model=schemas.User)
async def read_users_me(current_user: schemas.User = Depends(get_current_user)):
    return current_user

@router.get(
    "/{user_id}", response_model=schemas.User, dependencies=[Depends(get_current_user)]
)
async def read_user(user_id: int, db: Session = Depends(get_db)):
    db_user = crud.get_user(db, user_id)
    if db_user is None:
        raise HTTPException(
            status_code=status.HTTP_404_NOT_FOUND, detail="User not found"
        )
    return db_user

main.py

Add the following in main.py, It initializes the tables in the database:

from app.db.database import engine
from app.db import base

base.Base.metadata.create_all(bind=engine)

app = FastAPI()

Running the app with Auth and Users

Let's check our progress till now by running the app:

$ uvicorn app.main:app --reload

Try creating users, signing in with the credentials of the new user-created, and then retrieving the multiple users or a single one with all the path operations.

Creating Todo Items

Now, we will create CRUD operations for Todo Items

Create Todo Schemas

Let's start by creating Pydantic models for a to-do item.

Create a file [todo.py](http://todos.py) in the schemas/ folder:

from typing import Optional

from pydantic import BaseModel

class TodoBase(BaseModel):
    title: str
    description: Optional[str] = None
    done: bool = False

class TodoCreate(TodoBase):
    pass

class Todo(TodoBase):
    id: int
    owner_id: int

    class Config:
        orm_mode = True

Also, add the following line to __init__.py of schemas/: from .todo import Todo, TodoCreate

Create Todo SQLAlchemy Models

Create a file [todo.py](http://todo.py) inside models/:

from app.db.database import Base
from sqlalchemy import Boolean, Column, ForeignKey, Integer, String
from sqlalchemy.orm import relationship

class Todo(Base):
    __tablename__ = "todos"

    id = Column(Integer, primary_key=True, index=True)
    title = Column(String, nullable=False)
    description = Column(String)
    done = Column(Boolean, default=False)
    owner_id = Column(Integer, ForeignKey("users.id"))

    owner = relationship("User", back_populates="todos")

You are familiar with most of the things happening here, only the relationship thing is new.

A user and a todo are related to each other by the "owns" relationship such that a user owns a todo, we express this in a database using a foreign key that relates two entities together.

As one user can have multiple todo items but a todo item can have only one user associated with it, we placed this foreign key inside the todo item table.

The real magic lies in how easily we can access which user owns a todo item and all the todo items created by a user.

When accessing the attribute owner in a todo item, it will contain a User model from the users' table. SQLAlchemy will use the owner_id attribute/column with its foreign key to know which record to get from the users' table.

You also have to reflect this relationship inside the User model, so change it as follows:

from app.db.database import Base
from sqlalchemy import Column, Integer, String
from sqlalchemy.orm import relationship

class User(Base):
    __tablename__ = "users"

    id = Column(Integer, primary_key=True, index=True)
    username = Column(String, unique=True, nullable=False, index=True)
    hashed_password = Column(String, nullable=False)

    todos = relationship("Todo", back_populates="owner")

Now, accessing user.todos will give you a list of all the todo items created by that user.

Add the following line to __init__.py of models/: from .todo import Todo

Finally, update [base.py](http://base.py) in database/ to have the Todo model:

from app.db.database import Base
from app.models.users import User
from app.models.todo import Todo

Let's create a migration script and run the migrations to reflect the changes in the database:

$ alembic revision --autogenerate -m "Add todos table"

$ alembic upgrade head

You can check the database using sqlitebrowser, a new table would have been created for todo items.

Creating CRUD operations

We have to do the following operations on a todo item:

1. Create a new to-do item.

2. View all to-do items.

3. Mark a to-do item as completed.

4. Delete an existing to-do item.

Create a file [todo.py](http://todo.py) inside crud/.

1. Create Todo Item

    from app import models, schemas
    from sqlalchemy.orm import Session
   
    def create_todo(db: Session, todo: schemas.TodoCreate, user_id: int):
        db_todo = models.Todo(**todo.dict(), owner_id=user_id)
        db.add(db_todo)
        db.commit()
        db.refresh(db_todo)
        return db_todo
   

2. View all to-do items of current user

    def get_todos(db: Session, user_id: int, skip: int = 0, limit: int = 100):
        return db.query(models.Todo).filter(models.Todo.owner_id == user_id).offset(skip).limit(limit).all()
   

3. Mark a todo item as completed

    def update_todo_complete(db: Session, todo_id: int, complete: bool, user_id: int):
        db_todo = db.query(models.Todo).get(todo_id)
        if not db_todo:
            return None
        if db_todo.owner_id != user_id:
            raise Exception("Not authorized")
        db_todo.done = complete
        db.commit()
        db.refresh(db_todo)
        return db_todo
   

4. Delete an existing to-do item

    def delete_todo(
        db: Session,
        todo_id: int,
        user_id: int,
    ):
        db_todo = db.query(models.Todo).get(todo_id)
   
        if not db_todo:
            raise KeyError("Todo Not found")
   
        if db_todo.owner_id != user_id:
            raise Exception("Not authorized")
   
        db.delete(db_todo)
        db.commit()
   

These are the required four functions, let's add them to __init__.py of crud/: from .todo import create_todo, get_todos, update_todo_complete, delete_todo

Creating Router for To-do

It's time to create path operations for to-do items, create a file [todo.py](http://todo.py) inside routers/

from typing import List
from fastapi import APIRouter, Depends, HTTPException, status
from app import crud, schemas
from app.dependency import get_db, get_current_user
from sqlalchemy.orm import Session

router = APIRouter(prefix="/todos", tags=["todos"])

1. Path for creating to-do

    @router.post("/", response_model=schemas.Todo)
    async def create_todo(
        todo: schemas.TodoCreate,
        db: Session = Depends(get_db),
        current_user: schemas.User = Depends(get_current_user),
    ):
        todo = crud.create_todo(
            db,
            todo,
            current_user.id,
        )
        return todo
   

2. Path for viewing all todos

    @router.get("/", response_model=List[schemas.Todo])
    async def read_all_todos(
        skip: int = 0,
        limit: int = 100,
        db: Session = Depends(get_db),
        current_user: schemas.User = Depends(get_current_user),
    ):
        todos = crud.get_todos(db, current_user.id, skip, limit)
        return todos
   

3. Path for updating a to-do as complete/incomplete

    @router.put("/", response_model=schemas.Todo)
    async def update_todo_complete(
        todo_id: int,
        complete: bool,
        db: Session = Depends(get_db),
        current_user: schemas.User = Depends(get_current_user),
    ):
        try:
            todo = crud.update_todo_complete(
                db,
                todo_id,
                complete,
                current_user.id,
            )
        except Exception as e:
            raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail=str(e))
   
        if todo is None:
            raise HTTPException(
                status_code=status.HTTP_404_NOT_FOUND, detail="Todo not found"
            )
        return todo
   

4. Path for deleting a to-do item

    @router.delete("/{todo_id}")
    async def delete_todo(
        todo_id: int,
        db: Session = Depends(get_db),
        current_user: schemas.User = Depends(get_current_user),
    ):
        try:
            crud.delete_todo(db, todo_id, current_user.id)
        except KeyError as e:
            raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(e))
        except Exception as e:
            raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail=str(e))
   
        return "Todo Deleted"
   

Let's add the router to main.py

from app.routers import users, todo

base.Base.metadata.create_all(bind=engine)

app = FastAPI()

app.include_router(users.router)
app.include_router(todo.router)

Let's run the app:

$ uvicorn app.main:app --reload

Testing

Yay! We have created an API that does what we want. But does it?

Let's write some tests for the CRUD operations and API routes to validate our code.

What is Testing?

Testing is the act of making sure everything runs as we expect it to.

Let's see a simple example:

def square(x):
    return x * x

We can test the above function by using an assert statement:

assert square(10) == 100

If the square function we wrote is working properly the assert would do nothing, but if the square function is defined incorrectly then an AssertionError is raised.

Testing in FastAPI

For testing our FastAPI app we will use the pytest library to make things easy.

Install the pytest library by executing the following command:

$ pip install -U pytest

Create a new folder tests in the app directory.

Now, create a file named [conftest.py](http://conftest.py) inside tests. This file will be used to create a temporary database and fixtures.

What are fixtures?

Fixtures are functions decorated with a @pytest.fixture decorator.

When pytest goes to run a test, it looks at the parameters in that test function’s signature and then searches for fixtures that have the same names as those parameters. Once pytest finds them, it runs those fixtures, captures what they returned (if anything), and passes those objects into the test function as arguments.

• pytest docs

One use case of a fixture is to clear a database after each test and create a new one before each test.

Let's first create a temporary database for running tests so that we don't accidentally change our actual database.

Add the following in the [conftest.py](http://conftest.py) file:

import pytest
from app.dependency import get_db
from sqlalchemy.orm import Session, sessionmaker
from app.db.database import Base
from app.main import app

SQLALCHEMY_DATABASE_URL = "sqlite:///./app/test.db"

engine = create_engine(
    SQLALCHEMY_DATABASE_URL, connect_args={"check_same_thread": False}
)
TestingSessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)

@pytest.fixture(scope="session", autouse=True)
def db_setup():
    Base.metadata.create_all(bind=engine)
    yield
    Base.metadata.drop_all(bind=engine)

def override_get_db():
    try:
        db = TestingSessionLocal()
        yield db
    finally:
        db.close()

@pytest.fixture(scope="session")
def db() -> Generator:
    yield from override_get_db()

app.dependency_overrides[get_db] = override_get_db

A lot is going on here, let's break it down:

• First, we created a new database using the same SQLAlchemy function we used before but this time we used a different SQLALCHEMY_DATABASE_URL. This will create a new database file test.db instead of using the previous todo_app.db.

• Then, we created a fixture:

    @pytest.fixture(scope="session", autouse=True)
    def db_setup():
        Base.metadata.create_all(bind=engine)
        yield
        Base.metadata.drop_all(bind=engine)
  

  • The scope of a fixture defines when it will be destroyed. Here are other possible values for scope:

    Fixtures are created when first requested by a test, and are destroyed based on their scope:

    • function: the default scope, the fixture is destroyed at the end of the test.

    • class: the fixture is destroyed during the teardown of the last test in the class.

    • module: the fixture is destroyed during the teardown of the last test in the module.

    • package: the fixture is destroyed during the teardown of the last test in the package.

    • session: the fixture is destroyed at the end of the test session.

  • The autouse=True means that this fixture will automatically be used for every test without explicitly mentioning the fixture name, it makes sense because we will use a database for all tests.

  • Then in the function, the part above yield will be executed before running any test, and the part after will run after all the tests are completed.

    Before yield, we are creating a new database, and after yield, we are clearing that database.

• Then we created a new dependency that will use the temporary database. We override the get_db dependency with this one so any function in our app that uses the get_db will now use override_get_db, as a result, using the temporary database. This allowed us to use a new database without changing any code in our app.

• We also created another fixture db to access the database directly without the dependencies. This will help write tests for CRUD operations.

These things will be more clear when we use them in our tests.

Let's add three more fixtures to the conftest.py:

from typing import Dict, Generator
from app import models
from fastapi.testclient import TestClient
from . import utils    # utils.py is created next

@pytest.fixture(scope="module")
def client() -> Generator:
    with TestClient(app) as c:
        yield c

@pytest.fixture(scope="function")
def user(db: Session) -> models.User:
    return utils.create_random_user(db)

@pytest.fixture(scope="module")
def user_token_headers(client: TestClient, db: Session) -> Dict[str, str]:
    return utils.authentication_token(client=client, db=db)

Let's go through each fixture:

• The client fixture: We will use this to make API calls from our code like the get and post request. It is based on Python's request library.

• The user fixture: It will create random users to test the todo CRUD operations.

• the user_token_headers fixture: It is used to provide bearer token while accessing endpoints which needs authentication.

A few fixtures above uses functions from the [utils.py](http://utils.py) file, let's define it in the same tests folder:

import random
import string
from typing import Dict
from app import crud, models, schemas

from sqlalchemy.orm import Session
from fastapi.testclient import TestClient

def random_lower_string() -> str:
    return "".join(random.choices(string.ascii_lowercase, k=random.randint(1, 32)))

def create_random_user(db: Session) -> models.User:
    username = random_lower_string()
    password = random_lower_string()
    user_in = schemas.UserCreate(username=username, password=password)
    user = crud.create_user(db, user=user_in)
    return user

def user_authentication_headers(
    *, client: TestClient, username: str, password: str
) -> Dict[str, str]:
    data = {"username": username, "password": password}
    r = client.post("/token", data=data)
    response = r.json()
    auth_token = response["access_token"]
    headers = {"Authorization": f"Bearer {auth_token}"}
    return headers

def authentication_token(*, client: TestClient, db: Session) -> Dict[str, str]:
    username = random_lower_string()
    password = random_lower_string()
    crud.create_user(db, user=schemas.UserCreate(username=username, password=password))
    return user_authentication_headers(
        client=client, username=username, password=password
    )

Here's what each function is doing:

• random_lower_string: Creates a random string of max length 32 to be used in the test to create users and todo items.

• create_random_user: Creates and returns a random user.

• user_authentication_headers: Returns the authorization header to be used to authenticate API requests made through tests.

• authentication_token: Creates a new user and returns its authorization header.

Tests for CRUD operations

We are now done with the setup required to test our code, now we will test the CRUD operations for users and todos.

Create folder crud inside tests.

Create a file called test_user.py inside crud. Remember to prepend the file name by test_, this is how pytest will find test files to execute.

Let's define the test functions for each of the crud operation on user one by one:

Import the necessary modules

from app import crud
from app.schemas import UserCreate
from app.tests import utils
from sqlalchemy.orm import Session
from app.tests import utils
from app import models

Test creating a user

def test_create_user(db: Session):
    username = utils.random_lower_string()
    password = utils.random_lower_string()
    user_in = UserCreate(username=username, password=password)
    user = crud.create_user(db, user=user_in)
    assert user.username == username
    assert hasattr(user, "hashed_password")

This test function corresponds to the create_user function of crud/users.py. We are creating a user and then making sure the created user has the same username as the one used to create that user and has the hashed_password attribute.

Remember each test function's name must start with test_ and a test function's name should be descriptive.

Test authenticating a user

def test_authenticate_user(db: Session):
    username = utils.random_lower_string()
    password = utils.random_lower_string()
    user_in = UserCreate(username=username, password=password)
    user = crud.create_user(db, user=user_in)
    authenticated_user = crud.users.authenticate_user(
        db, username=username, password=password
    )
    assert authenticated_user
    assert authenticated_user.username == user.username

This function corresponds to the authenticate_user function of the crud/users.py. We are making sure an existing user with the correct credentials is authenticated.

Test not authenticating a user

def test_not_authenticate_user(db: Session):
    username = utils.random_lower_string()
    password = utils.random_lower_string()
    user_in = UserCreate(username=username, password=password)
    user = crud.create_user(db, user=user_in)
    assert not crud.authenticate_user(db, username=username, password="wrong")
    assert not crud.authenticate_user(db, username="wrong", password=password)
    assert not crud.authenticate_user(db, username="wrong", password="wrong")

This function also corresponds to the authenticate_user, but this time it is checking a different path that is when a user does not exists or enter incorrect credentials.

Test getting a user by id

def test_get_user_by_id(db: Session, user: models.User):
    user_out = crud.get_user(db, user_id=user.id)
    assert user_out
    assert user_out.username == user.username
    assert user_out.hashed_password == user.hashed_password

We are testing crud.get_user here, first we created a new user and called get_user with its id and checked if we are getting the correct user in output or not.

We also used the userfixture in the arguments**,** so the user parameter will be provided with a random user by pytest.

Test getting a user by username

def test_get_user_by_username(db: Session, user: models.User):
    user_out = crud.get_user_by_username(db, username=user.username)
    assert user_out
    assert user_out.username == user.username
    assert user_out.hashed_password == user.hashed_password

This one is similar to the previous one.

Testing getting all users

def test_get_users(db: Session, user: models.User):
    user_out = crud.get_users(db)
    print(user_out)
    assert user_out
    assert user_out[-1].username == user.username
    assert user_out[-1].hashed_password == user.hashed_password

Here, we are creating a new user using the fixture and getting all the users, and finally checking that the new user is the last one in the list or not.

That's it for test for user's crud operation.

Try running them with the following command in the todo directory:

$ python -m pytest app/tests/crud

You will get an output like this:

Each green dot represents a single test function and means that the test passed, an 'F' means that the test failed.

Here are the test functions for todo's crud operations(crud/todo.py), you will be able to understand them now:

Create a file test_todo.py in the tests/crud folder and the following tests:

import pytest
from app import crud, models, schemas
from app.tests import utils
from sqlalchemy.orm import Session

def test_create_todo(db: Session, user: models.User) -> None:
    """Test creating a todo"""
    todo_in = schemas.TodoCreate(
        title=utils.random_lower_string(),
        description=utils.random_lower_string(),
        done=False,
    )
    todo_out = crud.create_todo(db=db, todo=todo_in, user_id=user.id)
    assert todo_out.title == todo_in.title
    assert todo_out.owner_id == user.id

def test_get_todos(db: Session, user: models.User) -> None:
    """Test getting all todos"""
    todo_in_1 = schemas.TodoCreate(title=utils.random_lower_string())
    crud.create_todo(db=db, todo=todo_in_1, user_id=user.id)
    todo_in_2 = schemas.TodoCreate(title=utils.random_lower_string())
    crud.create_todo(db=db, todo=todo_in_2, user_id=user.id)
    todos = crud.get_todos(db=db, user_id=user.id)
    assert len(todos) == 2
    assert todos[0].title == todo_in_1.title
    assert todos[1].title == todo_in_2.title

def test_update_todo_complete(db: Session, user: models.User) -> None:
    """Test updating a todo's complete status"""
    todo_in = schemas.TodoCreate(title=utils.random_lower_string())
    todo = crud.create_todo(db=db, todo=todo_in, user_id=user.id)
    todo_out = crud.update_todo_complete(
        db=db, todo_id=todo.id, complete=True, user_id=user.id
    )
    assert todo_out.done is True

def test_update_todo_complete_when_todo_does_not_exist(
    db: Session, user: models.User
) -> None:
    """Test updating a todo's complete status when todo doesn't exist"""
    todo_in = schemas.TodoCreate(title=utils.random_lower_string())
    todo = crud.create_todo(db=db, todo=todo_in, user_id=user.id)
    todo_out = crud.update_todo_complete(
        db=db, todo_id=todo.id + 1, complete=True, user_id=user.id
    )
    assert todo_out is None

def test_update_todo_complete_when_todo_is_not_owned(
    db: Session, user: models.User
) -> None:
    """Test updating a todo's complete status when todo is not owned"""
    todo_in = schemas.TodoCreate(title=utils.random_lower_string())
    todo = crud.create_todo(db=db, todo=todo_in, user_id=user.id)
    with pytest.raises(Exception) as e:
        todo_out = crud.update_todo_complete(
            db=db, todo_id=todo.id, complete=True, user_id=user.id + 1
        )

def test_delete_todo(db: Session, user: models.User) -> None:
    """Test deleting a todo"""
    todo_in = schemas.TodoCreate(title=utils.random_lower_string())
    todo = crud.create_todo(db=db, todo=todo_in, user_id=user.id)
    crud.delete_todo(db=db, todo_id=todo.id, user_id=user.id)
    todos = crud.get_todos(db=db, user_id=user.id)
    assert todo not in todos

def test_delete_todo_when_todo_does_not_exist(db: Session, user: models.User) -> None:
    """Test deleting a todo when todo doesn't exist"""
    todo_in = schemas.TodoCreate(title=utils.random_lower_string())
    todo = crud.create_todo(db=db, todo=todo_in, user_id=user.id)
    with pytest.raises(KeyError) as e:
        crud.delete_todo(db=db, todo_id=todo.id + 1, user_id=user.id)
    todos = crud.get_todos(db=db, user_id=user.id)
    assert todo in todos

def test_delete_todo_when_todo_is_not_owned(db: Session, user: models.User) -> None:
    """Test deleting a todo when todo is not owned"""
    todo_in = schemas.TodoCreate(title=utils.random_lower_string())
    todo = crud.create_todo(db=db, todo=todo_in, user_id=user.id)
    with pytest.raises(Exception) as e:
        crud.delete_todo(db=db, todo_id=todo.id, user_id=user.id + 1)
    todos = crud.get_todos(db=db, user_id=user.id)
    assert todo in todos

Try running them.

Now we will write and understand test functions for the API.

Tests for API endpoints

Create folder api inside tests and create a file test_users.py.

Import necessary modules intest_users.py

from typing import Dict

from app import models, schemas
from fastapi.testclient import TestClient
from app.tests import utils

Test Create User

def test_create_user(client: TestClient):
    user = schemas.UserCreate(username="test", password="test")
    response = client.post("/users/", json=user.dict())
    assert 200 <= response.status_code < 300
    created_user = response.json()
    assert created_user["username"] == user.username

Here we are testing the /users/ endpoint when accessed with a post request. We are using the client fixture to make the request, the first parameter to [client.post](http://client.post) defines the path and the json parameter includes the json data needed to create a user.

Then we are asserting a successful response and creation of a correct user.

Here are a few ways to send data in a request using the client

• To pass a path or query parameter, add it to the URL itself.

• To pass a JSON body, pass a Python object (e.g. a dict) to the parameter json.

• If you need to send Form Data instead of JSON, use the data parameter instead.

• To pass headers, use a dict in the headers parameter.

• For cookies, a dict in the cookies parameter.

Remaining test functions for User's endpoint

Similarly, we can create tests for other endpoints:

def test_create_user_when_exist(client: TestClient):
    user = schemas.UserCreate(username="existing", password="existing")
    response = client.post("/users/", json=user.dict())
    assert 200 <= response.status_code < 300
    response = client.post("/users/", json=user.dict())
    assert 400 <= response.status_code < 500

def test_read_users(client: TestClient, user_token_headers: Dict[str, str]):
    user1 = schemas.UserCreate(
        username=utils.random_lower_string(), password=utils.random_lower_string()
    )
    response = client.post("/users/", json=user1.dict())
    assert 200 <= response.status_code < 300

    user2 = schemas.UserCreate(
        username=utils.random_lower_string(), password=utils.random_lower_string()
    )
    response = client.post("/users/", json=user2.dict())
    assert 200 <= response.status_code < 300

    response = client.get("/users/", headers=user_token_headers)
    assert 200 <= response.status_code < 300
    users = response.json()
    assert len(users) > 0
    assert users[-1]["username"] == user2.username

def test_read_user_me(client: TestClient, user_token_headers: Dict[str, str]):
    response = client.get("/users/me", headers=user_token_headers)
    assert 200 <= response.status_code < 300
    current_user = response.json()
    assert current_user
    assert "username" in current_user

def test_read_user(
    client: TestClient, user_token_headers: Dict[str, str], user: models.User
):
    user_id = user.id
    response = client.get(f"/users/{user_id}", headers=user_token_headers)
    assert 200 <= response.status_code < 300
    existing_user = response.json()
    assert existing_user
    assert existing_user["username"] == user.username

def test_read_user_when_not_found(
    client: TestClient, user_token_headers: Dict[str, str], user: models.User
):
    response = client.get(f"/users/{user.id + 1}", headers=user_token_headers)
    assert response.status_code == 404

Also, create a file test_todos.py in api and add the following tests:

from typing import Dict
from app import schemas
from fastapi.testclient import TestClient
from app.tests import utils

def test_create_todo(client: TestClient, user_token_headers: Dict[str, str]):
    todo_in = schemas.TodoCreate(
        title=utils.random_lower_string(), description=utils.random_lower_string()
    )
    response = client.post("/todos/", json=todo_in.dict(), headers=user_token_headers)

    assert 200 <= response.status_code < 300

    todo = response.json()
    assert todo["title"] == todo_in.title

def test_read_all_todos(client: TestClient, user_token_headers: Dict[str, str]):
    todo1 = schemas.TodoCreate(
        title=utils.random_lower_string(), description=utils.random_lower_string()
    )
    response = client.post("/todos/", json=todo1.dict(), headers=user_token_headers)
    assert 200 <= response.status_code < 300

    todo2 = schemas.TodoCreate(
        title=utils.random_lower_string(), description=utils.random_lower_string()
    )
    response = client.post("/todos/", json=todo2.dict(), headers=user_token_headers)
    assert 200 <= response.status_code < 300

    response = client.get("/todos/", headers=user_token_headers)
    assert 200 <= response.status_code < 300

    todos = response.json()
    assert len(todos) > 0
    assert todos[-1]["title"] == todo2.title

def test_update_todo_complete(client: TestClient, user_token_headers: Dict[str, str]):
    todo_in = schemas.TodoCreate(title=utils.random_lower_string())
    response = client.post("/todos/", json=todo_in.dict(), headers=user_token_headers)
    assert 200 <= response.status_code < 300

    todo = response.json()
    assert todo["title"] == todo_in.title

    response = client.put(
        f"/todos/?todo_id={todo['id']}&complete=true", headers=user_token_headers
    )
    assert 200 <= response.status_code < 300

    todo = response.json()
    assert todo["title"] == todo_in.title
    assert todo["done"] is True

def test_update_todo_complete_when_not_found(
    client: TestClient, user_token_headers: Dict[str, str]
):
    todo_in = schemas.TodoCreate(title=utils.random_lower_string())
    response = client.post("/todos/", json=todo_in.dict(), headers=user_token_headers)
    assert 200 <= response.status_code < 300

    todo = response.json()
    assert todo["title"] == todo_in.title

    response = client.put(
        f"/todos/?todo_id={todo['id'] + 1}&complete=true", headers=user_token_headers
    )
    assert response.status_code == 404

def test_delete_todo(client: TestClient, user_token_headers: Dict[str, str]):
    todo_in = schemas.TodoCreate(title=utils.random_lower_string())
    response = client.post("/todos/", json=todo_in.dict(), headers=user_token_headers)
    assert 200 <= response.status_code < 300

    todo = response.json()
    assert todo["title"] == todo_in.title

    response = client.delete(f"/todos/{todo['id']}", headers=user_token_headers)
    assert 200 <= response.status_code < 300

    response = client.get("/todos/", headers=user_token_headers)
    assert 200 <= response.status_code < 300

    todos = response.json()
    assert todo not in todos

def test_delete_todo_not_found(client: TestClient, user_token_headers: Dict[str, str]):
    todo_in = schemas.TodoCreate(title=utils.random_lower_string())
    response = client.post("/todos/", json=todo_in.dict(), headers=user_token_headers)
    assert 200 <= response.status_code < 300

    todo = response.json()
    assert todo["title"] == todo_in.title

    response = client.delete(f"/todos/{todo['id'] + 1}", headers=user_token_headers)
    assert response.status_code == 404

As an exercise, I encourage you to define a test function for the token/ endpoint from the [main.py](http://main.py) file.

With this we are done with testing, let's check the result.

Testing Result

Testing Coverage: This is a technique to determine how much application code we are testing.

Install the pytest-cov package which is a pytest plugin to measure test coverage.

$ install pip install pytest-cov

And run the following command in the todo directory:

$ python -m pytest app --cov=app --cov-report term-missing

We get the following output:

---------- coverage: platform linux, python 3.8.10-final-0 -----------
Name                          Stmts   Miss  Cover   Missing
-----------------------------------------------------------
...
app/crud/todo.py                 28      0   100%
app/crud/users.py                23      0   100%
...
app/routers/todo.py              32      4    88%   49-50, 69-70
app/routers/users.py             25      0   100%
...
-----------------------------------------------------------
TOTAL                           519     15    97%

=============== 26 passed, 6 warnings in 7.53s ================

We passed all 26 tests that we wrote and the test coverage is 97%.

Now try running the app with uvicorn and try out different endpoints yourself, try creating a user, signing in, creating and deleting todo items.

Woohoo!

Congratulations on making it this far.

We have created a Todo API with FastAPI and in the process learned a lot of things, I hope you have enjoyed it.

What's next?

I would recommend you to read the official docs of FastAPI and create your own Project.

FastAPI docs are one of the best I have seen with proper examples, even this article is highly inspired by the same and uses many examples of the docs.

You would now find it much easier to work your way through the docs and create structured projects with FastAPI.

For the Hashnode Christmas Hackathon, I decided to build a Twitter bot that does two things -

• Retweet every tweet with the following keyword in itself -
  • Hashnode
  • HashnodeCommunity
  • HashnodeBot

• Reply with the requested article when mentioned in a tweet.

  • Tweeting

    @HashnodeBot featured

    will get you a reply from the bot with today's two featured articles.

  • Tweeting

    @HashnodeBot [n]

    will get you a reply from the bot n last articles from the specified user.

Development

On December 28, Morning I started the project, I had never created a bot before so I searched google and learned about Tweepy) and how to use it.

Tweepy is a Python library for accessing the Twitter API.

Strategy

Now, was the time to decide how I will achieve the two requirements -

• Retweeting tweets with concerning keywords.
• Replying with Hashnode articles as demanded by the user.

The first task was easy and direct to do with Tweepy, I just had to write a simple Python script that waits for new tweets with given keywords and then retweet and like them if not done yet.

The second task required me to use both Hashnode and Twitter API, the idea was to read tweets that mention the bot @HashnodeBot and then using the Hashnode API get those tweet, but the problem was Hashnode uses GraphQL and I didn't understand it, So I gave a quick look to GraphQL documentation and figured out how to use it. (It was not that hard though 😅).

I wrote two python scripts for this task, one fetches articles using hashnode API, and the other replies to the tweet with Bot's mention.

Result

Here is how Retweet Bot looks like on Twitter -

and this is how Reply Bot looks like on Twitter -

Problem

For some reason, Twitter was not showing the replies from the bot to other users, only the reply count is increasing and the replies are visible from the bots account only.

Twitter was restricting the visibility of the Bot

I tried to get around this, searched Google but couldn't make it, The only reason I got is that Twitter spam filtering was responsible for this, so I decided to deploy the bot with only the retweet feature.

Deployment

I used Docker to deploy the bot on AWS, this was my first time with both Docker and AWS.

I also used the scp command for the first time to transfer the docker package to AWS.

Final Result

So, in the end, I created and deployed a Twitter Bot that retweets tweets with the following keywords - Hashnode, HashnodeCommunity.

In the first 24 hours of deployment, the bot has retweeted over 100 hashnode tweets.

You can follow the bot @HashnodeBot to stay updated with all the amazing articles on hashnode and support the writers.

Here are the resources I used in this project

• Realpython guide to creating Twitter bot
• Hashnode API
• Twitter
• Tweepy

Thanks For Reading

If you find this article and project helpful, please like and share, and leave your feedback in the comments, It means a lot. 🙏🏼

You may also like:

• The Mutable Default Argument Mess in Python)
• What Happens When You Run a Computer Program?
• Comprehensions in Python: Explained
• Linux Commands Reference With Examples

In this blog, we are going to find the answers to these questions -

• What is the internet?
• What is a network?
• What are IP addresses?
• What happens when you search on Google?
• What are the rules of the Internet?
• Where is Google’s server located?
• What devices hold the internet together?

Fasten your seat belts for a ride over routers...

What is the Internet?

• The Internet is the global system of interconnected computer networks that uses the Internet protocol suite (TCP/IP) to communicate between networks and devices.
• It is a network of networks.
  • Several collections of computers together make the internet.
• It is used for sharing various resources such as web pages, emails, file sharing.
• To understand the internet, we first need to understand what a network is?

What is a Network?

• A network is a collection of computers, servers, mainframes, network devices, peripherals, or other devices connected to one another to allow the sharing of data.

• Consider an example of a network: your home network -

  • You have a router in your home.
    • This router connects to the Internet.
    • You pay your isp for this connection - Airtel, Jio, etc.
    • Other devices connect to the router with cables or wifi.

What are IP addresses?

• Every device on the internet has an IP address that allows other computers to talk to it.
• It is of the form: #.#.#.#
  • Four numbers separated by dots of the values 0-255.
  • This version of IP addresses is called IPv4.
• Like postal addresses, they uniquely identify computers on the internet.
• ISPs assign an IP address to your router.
  • Earlier, it is used to be physically configured.
  • Today, it is assigned automatically using a DHCP Server that lives in ISP.
• But, if your ISP only assigns one address to the router, how are you able to connect multiple devices to the internet?
  • That’s because your router assigns private IP addresses to the devices.
  • This private IP address is free and reserved for this use only.
  • These are the private IP address ranges:
    • 10.0.0.0 – 10.255.255.255
    • 172.16.0.0 – 172.31.255.255
    • 192.168.0.0 – 192.168.255.255

What Happens When You Search on Google?

With an IP address assigned to us, we are officially on the internet. Now we can communicate with other devices on the internet. So, let's try to talk with Google Servers.

We will open software called a browser (like chrome) and enter a query to search with Google’s search engine. The browser will do the job of converting our normal query to a proper HTTP request to google.

• Suppose we searched for cats, our request will look like:

  GET /search?q=cats HTTP/1.1

Computers communicate by sending packets, which are like virtual envelopes sent between computers. (Ultimately still 0s and 1s).

So we will place this request inside an envelope (as an analogy). On the envelope, we need to mention the destination and source address.

• We know our IP address.
• But we don’t know the IP address of Google…?

But we do know the domain name of Google - www.google.com. To convert a domain name to its corresponding IP address, we use a Domain Name System(DNS) server.

A request will be sent to the ISP’s DNS server for Google’s IP address.

• If the ISP’s DNS server doesn’t know a website’s IP address, it has been configured to ask another DNS server.
• There exist root servers that know where to look to for an IP address if it exists.

We have assumed our IP to be 1.2.3.4 and google’s 5.6.7.8. After sending the request off, we’ll get a response some milliseconds later. The result will be sent back in one or more packets.

• If the result is too large for a single envelope, sending it in one packet could take up the internet traffic.
• To solve this, Google will divide the result into smaller fragments (say 4).
  • Put the fragments into different envelopes.
  • Write information on the envelopes:
    • Return address: Google IP address
    • Delivery address: Our IP address.
    • List the number of packets on each envelope (1 of 4, 2 of 4, …).

What if one of the packets goes missing?

We can logically infer which packet is missing based on the ones received. For this, we will need another protocol called TCP (Transmission control protocol) which ensures packets can get to their destination.

It supports sequence numbers that help data get to its destination.

• When missing a packet, a computer can make a request for the missing packet.
• The computer will put packets together to get a whole file.

This protocol also includes conventions for requesting services (port identifiers).

• To make sure Google knows we’re requesting a web page and not an email or other service.

If 5.6.7.8 is Google’s IP address, 5.6.7.8:80 (port 80) specifies that we want a webpage.

• 80 means HTTP (Hypertext transfer protocol).
  • The language web servers speak.
• Emails use port 25 - SMTP.
• Secure connection use 443 - HTTPS.

Therefore, on the envelope we previously saw, we must have mentioned the port as well before sending it.

What are the Rules of the Internet?

We have used the term protocols multiple times till now. Protocols are just sets of rules.

• Humans use these all the time, such as the protocol for meeting people: handshakes. (pre-corona times!!)

Internet uses these protocols to standardize all communication.

• HTTP, FTP, TCP/IP, etc.

There is one more protocol called UDP (User datagram protocol) that can be used instead of TCP.

• It doesn’t guarantee delivery.

• Used for video conferencing.

  • Packets can be dropped for the sake of keeping the conversation flowing.

• Used anytime you want to keep data coming without waiting for a buffer to fill.

Where is Google's Server Located?

When we sent our envelope, our laptop didn’t know where is the Google’s server located. We knew the IP address of the server but we didn’t have a map to decide which direction to go to and we certainly cannot use Google maps 😅.

This again is the job of routers:

• Routers have a big table with IP addresses and where data should be routed to get to that destination. Often, the data is routed to the next router.

The router’s purpose is to send data in the direction of a destination. The next router will send it to another until it reaches a destination.

The routing decisions are made using certain routing algorithms such as Dijkstra’s shortest path algorithm or distance vector routing algorithm.

We can use a program called traceroute that sends packets to each router on a path to a destination, reporting the time it takes to reach that router:

• It took us 13 jumps to get to google finding our way through routers.
• Also, Now we can see Google’s IP address is not 5.6.7.8 but 142.250.67.68.
• It took us only 42.130 ms to get to Google!!!

What Devices Hold the Internet Together?

• Undersea Cables: A submarine communications cable is a cable laid on the sea bed between land-based stations to carry telecommunication signals across stretches of ocean and sea.

  Check out this video to see the network of undersea cables around the world:

  https://youtu.be/IlAJJI-qG2k

• Cable Modems: A cable modem is a hardware device that allows your computer to communicate with an Internet service provider over a landline connection. It converts an analog signal to a digital signal for the purpose of granting access to broadband Internet.

  • Coaxial cable to plug into the wall.
  • Phone jacks (RJ11) as many services are bundled together these days.
  • Four jacks for ethernet cables (RJ45).
  • Devices can plug into these for internet connectivity.

• Switch: A network switch is networking hardware that connects devices on a computer network by using packet switching to receive and forward data to the destination device.

• Router: A router is a networking device that forwards data packets between computer networks. Routers perform the traffic directing functions on the Internet. Home routers can have wifi, firewall, and switching capabilities.

Thanks For Reading 😊

If you found this article helpful, please like and share! Feedbacks are welcome in the comments.

You may also like:

• What Happens When You Run a Computer Program?
• The Mutable Default Argument Mess in Python
• Modern C++ Features
• Comprehensions in Python: Explained

Connect with me on Twitter, GitHub, and LinkedIn.

This article is inspired by a presentation I gave with my friend Manmeet Brar in my Computer Network class.

References: CS50 understanding technology.

Design a function that appends '#' to a list provided as an argument and then prints it. If no argument is provided then the function should use an empty list as a default.

Kevin quickly comes up with the following solution:

def append(l = []):
    l.append('#')
    print(l)

Looks good, time for testing the solution:

append([1, 2, 3])
# OUTPUT: [1, 2, 3, '#'] | OK

append()
# OUTPUT: ['#']    | OK

append()
# OUTPUT: ['#', '#']  | Strange!!

• The first call to append worked fine. It added # to the list [1, 2, 3] and then printed it.

• The second also worked as expected. This time no list is provided as an argument, so it used the default empty list and appended a # to it.

• Now, the third call resulted in something unexpected.

  When we once again called the append without argument, it printed ['#', '#'] instead of ['#'] as we got in the above call.

Why did this happened?

The reason this happened is that Python defines its default argument only once when the function is first defined.

This is because python is parsed line by line, when the parser encounters def it sets the default argument to a value and that value is used in every future call.

This behavior of Python becomes of special concern when the default argument is a mutable.

As the value of , immutables can not be changed, if you update the argument variable inside the function it will create a new object and start pointing to that object instead of changing the original default object.

But in the case of a mutable default argument, the object created at the time of parsing the function is updated instead of creating a different object for that function call.

Solution

The solution to this problem is to use an immutable default argument instead of a mutable. The preferred choice is None (though you can choose any immutable value).

def append(l = None):
    if l is None:
        l = []
    l.append('#')
    print(l)

Let's test this solution -

append([1, 2, 3])
# OUTPUT: [1, 2, 3, '#']    | OK

append()
# OUTPUT: ['#']     | OK

append()
# OUTPUT: ['#']        | Works fine!

Great! this solution worked as expected.

But why? Let's take a look inside..

Why the solution works?

Watch this video to find out what happened in the wrong version of the code -

As you can see, in this case, the original l was modified instead of creating a new l for each function call as a list is a mutable value.

Now, see the corrected version of the code -

Here as None is an immutable value therefore it cannot be changed and a new list object is created for each function call.

Note: The default argument is a property of the function object therefore, initially it is the same for all function calls.

Thanks For Reading 😊

If you found this article helpful, please like and share! Feedbacks are welcome in the comments.

You may also like:

• Modern C++ Features
• What Happens When You Run a Computer Program?
• Comprehensions in Python: Explained
• Linux Commands Reference With Examples

Connect with me on Twitter, GitHub, and LinkedIn.

In this article, I have shown a few features that you may know from Python or JavaScript that you can use in C++ too!!

This article is for people with a basic understanding of C++

Table of Contents

• Templates
• Auto
• Variable Argument Function
• Standard Template Library
• Range Based For Loop
• Lambda Expressions

Templates

Suppose you want to write a function with the same operations on different data types.

This is straightforward in Python as the same variable can point to different objects of different types. So you can reuse the same code for all types.

Now, in C++ one solution is to overload the function with different types but that will require you to write a lot of code that basically does the same thing. Here comes the use of templates.

Instead of writing functions with different types yourself, you can have the compiler generate them for you by using templates. The compiler knows which functions to generate based on the parameter provided.

Let's see an example:

• A template function for getting the maximum of two values, the value can be of any data type.

  #include <iostream>
  
  template <typename type>    // Creating a template
  type max(type first, type second)
  {
      return first > second ? first : second;
  }
  
  int main()
  {
      std::cout << max<int>(2, 3) << "\n";            // 3
      std::cout << max(3.4f, 4.2f) << "\n";    // 4.2
      std::cout << max('C', 'S') << "\n";        // S
  
      return 0;
  }
  

  • The first call will tell the compiler to create a function with type replaced by int
  • Similarly, the second and third calls will create a function with type replaced by float and char.
  • In the first call we have explicitly told the compiler that type is int with <type> notation.

You can similarly create templates for classes.

To learn more about templates in C++, checkout this article

Auto

You can deduce the type of a variable at compile time using the auto keyword.

This is useful when a variable has a complicated type.

Here is an example of auto

#include <iostream>
#include <string>

int main()
{
    auto an_int = 50;
    auto a_float = 10.5f;
    auto a_double = 100.12;
    auto a_char = 'S';
    auto a_string = "String";

    std::cout << an_int << "\n";        // 50
    std::cout << a_float << "\n";        // 10.5
    std::cout << a_double << "\n";        // 100.12
    std::cout << a_char << "\n";        // S
    std::cout << a_string << "\n";        // String

    return 0;
}

To learn more about auto in C++, read this article

Variable Argument Functions

If you know Python then you must have used * for creating a function with a variable number of arguments.

You can do the same in C++ using Ellipsis and variadic templates.

Let us understand this with an example -

#include <iostream>

template <typename type>
int sum(type n)
{
    return n;
}

template <typename first, typename... types>    // Packing Arguments
int sum(first n, types ...args)        // Expanding Arguments
{
    return n + sum(args...);    
}

int main()
{
    int sum3 = sum(1,2,3);
    std::cout << sum3 << "\n";        // 6
    int sum5 = sum(1,2,3,4,5);
    std::cout << sum5 << "\n";        // 15

    return 0;
}

Here, ... after typename is packing all the arguments passed when calling into types, and then we are unpacking them again with ... before args.

Then we have used recursion to call a function with arguments one less than the original.

Similarly you can create a print function that takes any number of arguments -

#include <iostream>

template <typename First>
void print(First first)
{
    std::cout << first << "\n";
}

template <typename First, typename ... Types>
void print(First first, Types ... args)
{
    std::cout << first << " ";
    print(args...);
}

int main()
{
    print("One", "Two");
    print("One", "Two", "Three", "Four");

    return 0;
}

Learn more about ellipsis and variadic templates in this article

Standard Template Library

In C++, only a few primitive data types are available inbuilt, unlike Python which provides a dynamic list, dictionary, set and tuples, etc.

Standard template library(STL) is a software library for C++, which provides these kinds of data structures and algorithms.

STL provides the following four components:

• Algorithms: searching, sorting, etc.
• Containers: Sequences such as vector, list. Queue, stack, map, set, etc.
• Functions
• Iterators

It is called a template library because it uses the C++ template feature discussed above.

Let's see one example of a container: vector.

Vector

The STL vector class is a class template for sequence containers.

It stores elements of a given type in a linear order and allows fast random access to any element. Sounds like an array? Yes, it is an array but it is dynamic means its size can change when needed.

This example shows how to create and use a vector:

#include <iostream>
#include <vector>    // STL Header file for vector class

int main()
{
    std::vector<int> numbers{1, 2, 3, 4, 5};

    for (int i = 0; i < numbers.size(); i++)
        std::cout << numbers[i] << " ";
    std::cout << "\n";  // OUTPUT: 1 2 3 4 5

    std::vector<char> alphabets;    // Empty vector
    alphabets.push_back('a');    // adding an element to back of vector
    alphabets.push_back('b');
    alphabets.push_back('c');

    std::cout << alphabets[1] << "\n";  // OUTPUT: b

    return 0;
}

To learn more about STL, checkout this article

Range Based For Loop

In the previous example, we used a simple for loop to iterate over our vector. We can do the same with the better range-based for loop.

Syntax:

for (for-range-declaration : expression)
    statement

So this is how we can change the previous code with a range based for loop:

// For printing the numbers vector
for (int x : numbers)
    std::cout << x << " ";

// For printing the alphabets vector
for (auto ch : alphabets)    // Notice the use of auto - Recommended
    std::cout << ch << " ";

for (auto i : {1, 2, 3})
        std::cout << i << " ";    // 1 2 3

int arr[] = {4, 5, 6};
for (auto j : arr)
    std::cout << j << " ";    // 4 5 6

Here x will point to an individual element in numbers one by one.

To learn more about range-based loops, read this article

Lambda Expressions

A lambda expression is a convenient way of defining an anonymous function.

Lambdas are used to encapsulate a few lines of code that are passed to algorithms and other functions.

Syntax:

1. capture clause: Specifies which variables are captured from the surrounding scope and whether the capture is by value or by reference.

   • Variables prefixed with & are accessed by reference other by value.

   • Empty clause [ ] indicates that the lambda body has no access to variables in the enclosing scope.

   • [&] means all variables are captured by reference.
   • [=] means all variables are captured by value.

2. parameter list (optional): A lambda can accept input parameters just like a normal function. It is optional and you can leave the parenthesis empty in case of no parameters.

3. mutable specification (optional): mutable keyword allows us to modify a value captured in the lambda body. (Usage is shown in the example below).

4. exception-specification (optional): You can use the noexcept exception specification to indicate that the lambda expression does not throw any exceptions.
5. trailing-return-type (optional): Just like normal function a lambda also have a return type. You can omit the return-type part of a lambda expression if the lambda body contains just one return statement or the expression does not return a value.
6. lambda body: A lambda body can contain anything that a normal function's body can contain.

These are a few examples of lambda functions:

• Here we have used a lambda function that finds the square of a number. The type of the lambda is automatically deduced using auto and it is stored in an identifier square.

  The return type is automatically deduced to int.

  #include <iostream>
  
  int main()
  {
      auto square = [](int n) {
        return n * n;  
      };
  
      std::cout << "Square of 2 = " << square(2) << "\n";
      // Square of 2 = 4
  
      return 0;
  }
  

• Here, we have shown the usage of the mutable keyword, without using it we would have received an error saying outer is read-only.

  Moreover, the value of the outer is unchanged outside lambda as it was captured by value [=].

  #include <iostream>
  
  int main()
  {
      int outer = 50;
  
      auto fn = [=]() mutable {
          outer = 100;    // OK, because of mutable
          std::cout << "Outer = " << outer << "\n";   // Outer = 100
      };
  
      fn();
      std::cout << "Outer = " << outer << "\n";   // Outer = 50
  
      return 0;
  }
  

• Here, we have used lambda to pass a function to the for_each algorithm.

  #include <iostream>
  #include <algorithm>
  #include <vector>
  
  int main()
  {
      std::vector<int> nums{1, 2, 3, 4, 5, 6};
  
      for (auto x : nums) // 1 2 3 4 5 6 
          std::cout << x << " ";
      std::cout << "\n";
  
      // Square every number
      std::for_each(begin(nums), end(nums), [](int& n) {n = n * n;}); 
  
      for (auto x : nums) // 1 4 9 16 25 36 
          std::cout << x << " ";
      std::cout << "\n";
  
      return 0;
  }
  

To learn more about lambda functions, checkout this article)

EndNote

This article was only supposed to introduce you to some modern powerful features provided by the C++ language.

If you want to learn more about these features, you may check out the links provided for each topic. The best way to learn these is by using it.

C++ has more amazing and modern features contrary to the belief that it is an old and outdated language.

Share your opinion in the comments.

Thanks For Reading

Read more on Scaler Topics

• Abstract Classes in C++
• Virtual Base Class in C++
• Difference Between C and C++
• Constructor and Destructor in C++

If you want to learn C++ from basic to advanced, do check out the C++ course from Scaler Topics.

You may also like:

• How to Create Header Files in C++
• What Happens When You Run a Computer Program?
• Comprehensions in Python: Explained
• Linux Commands Reference With Examples

Connect with me on Twitter, GitHub, and LinkedIn.

In this article, I have shown how to create your own header files in the right way.

This article is divided into two sections -

1. Creating Ordinary Header Files
2. Creating Template Header Files

Creating Ordinary Header Files

Suppose you have created the following header file with the following line in it-

const int SOME_CONSTANT = 50;    // first.h
// ...More

Now you have a second header file which has included the above file -

#include "first.h"    // second.h
// ...More

And you have one more C++ file which includes both these files -

#include "first.h"    // third.cpp
#include "second.h"    
// ...More

Now, when you compile your third.cpp file, you will get an error like this -

error: redefinition of ‘const int SOME_CONSTANT’ 1 | const int SOME_CONSTANT = 50;

Because the first.h file is included twice, therefore its content also gets copied twice in the third.cpp file.

To prevent this we should use Header Guards.

Using Header Guards

Header Guards are conditional compilations directives which mean they are evaluated at compilation time and the compiler will perform operations on the basis of its result.

Header Guards look like this -

#ifndef HEADERFILE_H
#define HEADERFILE_h

// Declarations goes here

#endif

ifndef means if not defined, if the HEADERFILE_H is not defined it will define it, otherwise if it is already defined it will skip compiling everything between ifndef and endif.

Now, we will update our previous example with header guards

• first.h

  #ifndef FIRST_H
  #define FIRST_H
  
  const int SOME_CONSTANT = 50;
  // ...More
  
  #endif
  

• second.h

  #ifndef SECOND_H
  #define SECOND_H
  
  #include "first.h"
  // ...More
  
  #endif
  

• third.cpp

  #include "first.h"
  #include "second.h"
  #include <iostream>
  
  int main()
  {
      // Using first.h and second.h
  }
  

Now, our third.cpp file will compile without error.

One Complete Example

Let's look at one more example -

In this example, we will create the following three files -

• headerfiles.h - Provides constants, functions and class declarations for use.
• definitions.cpp - Contains definitions for the headerfiles.h functions and classes.
• main.cpp - Consumes the functions and classes declared in headerfiles.

headerfile.h

#ifndef HEADERFILE_H
#define HEADERFILE_H

#include <iostream>
#include <string>

const int SOME_CONSTANT = 50;

void print(std::string message);

class SomeClass
{
    private:
        int var;

    public:
        SomeClass(int var);
        void print();
};

#endif

definitions.cpp

#include "headerfile.h"

void print(std::string message)
{
    std::cout << "Message = " << message << "\n";
}

SomeClass::SomeClass(int var)
{
    this->var = var;
}

void SomeClass::print()
{
    std::cout << "var = " << var << "\n";
}

main.cpp

#include "headerfile.h"
#include <iostream>

int main()
{
    print("Hello, World");  // OUTPUT: Message = Hello, World

    SomeClass object(50);

    object.print();     // OUTPUT: var = 50

    std::cout << SOME_CONSTANT << "\n"; // OUTPUT: 50

    return 0;
}

This is the right way to create header files, you separate the declarations and definitions in two files and only include the header file when necessary.

You will link the definitions file when compiling like this -

g++ -o main main.cpp definitions.cpp

Creating Template Header Files

Now in case of templates, you can't simply create two different files for declarations and definitions like the previous example because, when compilers compile the program it needs to replace the generic types with the required template arguments.

For example -

# foo.h
template <typename type>
type sum(type a, type b)
{
    type result = a + b;
    return result;
}

# main.cpp
sum(50, 100);    // This will create a function sum with type replaced with int everywhere

If we haven't provided the definition in the header file, the compiler won't be able to do perform the replacement(or instantiation).

Now, to separate our definition and declaration in the case of templates we can use the following way -

max.h

Header file to store function and class declaration

#ifndef MAX_H
#define MAX_H

template <typename type>
type max(type a, type b);

#include "max.tpp"    // IMPORTANT: We are including our definitions in the header file at the end
#endif

max.tpp

Definition file, note that its extension is .tpp and not .cpp as per the convention for template definitions files.

#include "max.h"

template <typename type>
type max(type a, type b)
{
    return a > b ? a : b;
}

main.cpp

#include "max.h"
#include <iostream>

int main()
{
    std::cout << max(50, 100) << "\n";   // 100

    return 0;
}

Thanks for reading

References:

• https://www.learncpp.com/cpp-tutorial/header-guards/
• https://stackoverflow.com/questions/495021/why-can-templates-only-be-implemented-in-the-header-file