Set python env variable windows

In this post, you’ll learn about how to use environment variables in Python on Windows, macOS, and Linux. Environment variables in Python enable you to write smoother workflows and more secure code.

You’ll learn why environment variables are useful to learn and how to implement them. You’ll learn how to get and set environment variables in Python. You’ll also learn how to implement Python environment variables safely in source control management tools like Github.

By the end of this tutorial, you’ll have learned:

• What environment variables in Python are and why you’ll want to use them
• How to get and set environment variables in Python using Windows, macOS, and Linux
• How to improve the environment variable workflow with the dotenv library
• How to safely store environment variables when working with source control management tools like Github

Python environment variables are variables that exist outside of your code and are part of your system’s or current user’s configurations. Why would you want to use environment variables? Environment variables provide the opportunity to make your code:

• More streamlined – by not needing to repeat constants such as API keys across multiple files. Similarly, using environment variables gives you the opportunity to configure your code so it can be tailored to a particular user running your code. For example, if your code is used by multiple people, the user’s path can be accessed from the environment.
• More secure – by not exposing secure keys or user configurations within your code. This allows you to share code that relies on API keys or other secure information without needing to expose that code. This prevents others from ready these secure pieces of information.

Environment variables represent key-value pairs of data that can be accessed by your code. As you’ll learn later, they can be accessed just like any dictionary value.

When To Use Python Environment Variables

Environment variables should be used when a variable changes with the environment (such as when a different user runs the code). This means that the best use cases are when the code would otherwise need manual updating when run in a different environment.

Similarly, environment variables should be used when secure data are implemented in a shared piece of code (or code shared to a source control management tool like Github). This allows you to embed secure information outside the code, while still being able to run your code.

When you first learn about environment variables, it may seem like a good idea to use them for everything. Following the guideline mentioned above sets a good standard for when to use them.

How to Get Environment Variables in Python

Environment variables can be accessed using the os library. In particular, they’re stored in the environ attribute. Because the os package is part of the standard Python library, you don’t need to install anything extra in order to get it to run.

In a later section, you’ll learn how to use the dotenv module to manage environment variables more easily. For now, let’s take a look at how you can get all environment variables using the os library.

How to See All Python Environment Variables

The easiest way to get all the environment variables available is to print out the os.environ attribute. In order to do this, you’ll need to first import the library. Let’s see what this looks like:

# Getting All Environment Variables Using os
import os
print(os.environ)

# Returns:
# environ({'TERM_PROGRAM': 'vscode', ...})

I have truncated the returned value above to make it more readable. Running this on your machine will likely display a large amount of variables.

We can check the type of this variable to see what it is:

# Checking the Type of the os.environ
import os
print(type(os.environ))

# Returns
# <class 'os._Environ'>

In the next section, you’ll learn how to get a single environment variable in Python.

How to Get a Single Environment Variable in Python

Because the returned value of the os.environ attribute is a dictionary-like structure, we can access a particular variable by using its key. For example, if we wanted to get the environment variable for 'USER' we could access it as we would any other dictionary’s value:

# Getting a Single Environment Variable in Python
import os
print(os.environ['USER'])

# Returns: datagy

Now, what would happen if we tried to access a variable that didn’t exist? Let’s try getting the environment variable for 'nonsense':

# Getting an Environment Variable that Doesn't Exist
import os
print(os.environ['nonesense'])

# Returns: KeyError: 'nonsense'

We can see that this raises a KeyError. If we don’t want our program to crash, we can use the .get() method to safely return None if no value exists. Let’s see what this looks like:

# Returning None if No Environment Variable Exists
import os
print(os.getenv('nonsense'))

# Returns: None

In the next section, you’ll learn how to check if an environment variable exists in Python.

How to Check if an Environment Variable Exists in Python

Because the returned value from the os.environ attribute is dictionary-like, you can use the in keyword to check whether or not an environment variable exists in Python. Let’s see how we can check if the variable 'USER' exists on the environment using an if-else block.

# Checking if an Environment Variable Exists in Python
import os

if 'USER' in os.environ:
    print('Environment variable exists!')
else:
    print('Environment variable does not exist.')

# Returns:
# Environment variable exists!

Using this conditional allows you to safely run code to see if an environment variable exists. If the variable doesn’t exist, you could, for example, prompt the user for input before continuing.

In the following section, you’ll learn how to return a default value if one doesn’t exist.

How to Return a Default Value for Environment Variables If One Doesn’t Exist

If no value exists for an environment variable, you may want to pass in a default value. This can also be done by using the .getenv() method. By passing in a second parameter, a default value can be returned if a variable doesn’t exist.

Let’s see what this looks like:

# Returning a Default Value When a Variable Doesn't Exist
import os
print(os.getenv('nonsense', 'default value'))

# Returns: default value

In the next sections, you’ll learn how to set environment variables.

How to Set Environment Variables in Python

Now that you know how to get environment variables using Python, in this section, you’ll learn how to set the values for new or existing variables. Because this process is different for Windows and macOS / Linux, the process is split across two different sections.

How to Set Environment Variables in Python Using macOS And Linux

To set an environment variable in Python using either macOS or Linus is to use the export command in a shell session. For example, if we wanted to set the variable of API_KEY to be equal to '123acb', we could write the following:

# Setting an Environment Variable
export API_KEY = '123abc'

When you run this code in your terminal, the environment variable will be set globally for all programs for that session. When you close your terminal, the environment variables are lost again.

If you only wanted to set the environment variable for a particular script, you can do this as well from the shell command. Simply change the command to this:

# Setting an Environment Variable
API_KEY = '123abc' python myscript.py

How to Set Environment Variables in Python Using Windows

Windows provides very similar ways to set environment variables in Python. Similar to macOS and Linux, you can set the variables for a single session by using the command prompt. Rather than using export, you use the word set. Let’s take a look:

# Setting an Environment Variable
set API_KEY = '123abc'

Instead, if you’re using the PowerShell console, you need to use the following code:

# Using PowerShell to Set an Environment Variable
$Env:API_KEY = '123abc'

In the following section, you’ll learn a much easier way to set and manage environment variables, using the dotenv library.

How to Use dotenv in Python to Work with Environment Variables in Python

Using the terminal or command prompt to manage environment variables can be a time-consuming and, frankly, annoying task. Because of this, Python provides the option to use the dotenv library to better manage environment variables.

The benefit of using the dotenv library is that the process is the same for Windows, macOS, and Linux. The way that this process works is by including a .env file in the project directory that you’re working in. The file will contain the environment variables that you’re hoping to use in your script.

Let’s see how we can use the dotenv library to manage our environment variables better in Python. First, we’ll need to install the dotenv library, which can be done using either pip or conda:

# Install dotenv
pip install python-dotenv
conda install python-dotenv

Once this is done, we need to create the file in the directory. The easiest way to do this is using the terminal:

touch .env

Once this is done, you can open the file, which is really just a text file. From there, you can add all of the environment variables that you want to use. The way that this is done is by placing different variables on new lines and creating the key-value pair by using an equal sign =:

API_KEY=123abc
user_name=Nik

From there, in your Python script it’s as easy as importing the library and loading the file:

# Loading an Environment Variable File with dotenv
from dotenv import load_dotenv
load_dotenv()

The function will search through the current directory for the .env file. If it doesn’t find one, then it continues to move up through the directories to try and find the file.

Similarly, you could simply pass in the full path to the file to make your code more explicit:

# Loading an Environment Variable File Explicitly with dotenv
from dotenv import load_dotenv
load_dotenv('/home/datagy/project/.env')

Using dotenv Safely with Source Control Management Tools

Because environment variables can be used to store sensitive information, it’s important to be mindful of not including them when working with SCM tools like Github. In order to work with these safely, you can add them to a .gitignore file.

In order to make your code more understandable for others, simply add an example .env file to your repository. This allows readers of your code to understand what environment variables must be set.

Conclusion

In this tutorial, you learned all about environment variables in Python. Environment variables provide the ability to elegantly and securely run your code. They provide the opportunity to streamline your code without repeating and changing your code. Similarly, they prevent having to share secure items such as API Keys.

You learned how to get and set environment variables in Python using the os library. You also learned how to use the dotenv library to make managing environment variables much easier in Python.

Additional Resources

To learn more about related topics, check out the tutorials below:

• Use Python to List Files in a Directory (Folder) with os and glob
• Python: Get and Change the Working Directory
• Python: Check if a File or Directory Exists
• Python Delete a File or Directory: A Complete Guide
• Python dotenv Official Documentation

As a modern application, yours always deals with credentials, secrets, and configurations to connect to other services like an authentication service, database, cloud services, microservices, etc.

Keeping your username, password, and other login information in the source code of your application is not a good idea because they could be exposed if you share or publish the programme.

Before sharing the code, you must delete or remark the credentials, which requires more work from you. Also, you might eventually forget to do it. The credentials you provide to the application via the command line parameters may be visible to others as well as to you.

The source code should not also contain hard codes for the setups of the services, such as API endpoints and database URLs. The issue is that you must rewrite the code each time you change or update the configurations, which could result in further mistakes.

How should we solve this issue using environment variables?

Let’s deal with it.

• What are Environment Variables?
• Types of Environment Variables
  1. User Environment Variables
  2. System Environment Variables
  3. Virtual Environment Variables
• Accessing environment variables using Python
  1. How to get an environment variable in Python?
  2. How to set an environment variable in Python?
  3. How to delete an environment variable in Python?
  4. How to retrieve list of Environment Variables?
• Python environment variables examples
• Setting Environment Variables with .env file
• Use cases of Environment Variables

What are Environment Variables?

The variable that is connected to an environment is referred to as an environment variable. In other words, the external variables you keep outside of your program will affect how your application behaves.

Environment variables are variables that are dynamically available to your programme or application while it is running. Values of these variables may originate from text files, outside secret managers, calling scripts, etc.

The most important fact is the value of these environment variables is not hardcoded into your application. They are genuinely dynamic and adaptable to the setting in which your software is running.

Example:

When I use my computer to work, it is my environment. It has a few variables that I may utilise on my machine globally. If you use Windows, you can search for environment variables in system properties and a list of them will then show.

It contains user and system environment variables and key(variable name)  value(variable value) pairs of all the environment variables. Here, the PATH is a general environment variable used by OS, specifying a set of directories where executable programs are located.

What is an .env file?

A text file called a .env file stores key-value pairs for each environment variable your application needs. You can use environment variables for local development using a .env file without contaminating the global environment namespace.

It’s an easy approach to save global variables that you need but doesn’t want publicly available online such as storing private data. For example:

• Database login information
• Username and password
• API tokens
• Secret codes
• Crypto Wallet keys

Here is an example of a typical .env file:

VAR_UNO = SOME_KEY_HERE
VAR_DOS = SOME_OTHER_KEY_HERE

Here, VAR_UNO,VAR_DOS are keys and SOME_KEY_HERE, SOME_OTHER_KEY_HERE are values.

Types of Environment Variables

In Python, there are three types of environment variables that can be used to configure a program:

1. User environment variables

These are variables that are set by the user on their system. They are specific to the user and are not visible to other users of the system. In Python, you can access user environment variables using the os.environ dictionary.

import os

# Get the value of a user environment variable
my_var = os.environ.get('MY_VAR')

# Set the value of a user environment variable
os.environ['MY_VAR'] = 'my_value'

In this example, we use the os.environ dictionary to get and set the value of a user environment variable named «MY_VAR». If the variable is not set, os.environ.get() will return None.

2. System environment variables

These are variables that are set at the system level and are available to all users of the system. They are typically used to store configuration information that is needed by many different programs. In Python, you can access system environment variables using the os.environ dictionary.

import os

# Get the value of a system environment variable
my_var = os.environ.get('PATH')

# Set the value of a system environment variable (not recommended)
os.system('export MY_VAR=my_value')

In this example, we use the os.environ dictionary to get the value of a system environment variable named «PATH». System environment variables are set at the system level, so we can’t set them directly in Python. Instead, we can use the os.system() function to execute a shell command that sets the variable. However, it is generally not recommended to set system environment variables this way.

3. Virtual environment variables

These are variables that are specific to a virtual environment created by a tool like virtualenv. Virtual environments allow you to create a self-contained environment that has its own set of dependencies and configurations. In Python, you can access virtual environment variables using the os.environ dictionary, just like user and system environment variables.

import os

# Get the value of a virtual environment variable
my_var = os.environ.get('MY_VAR')

# Set the value of a virtual environment variable
os.environ['MY_VAR'] = 'my_value'

In this example, we use the os.environ dictionary to get and set the value of a virtual environment variable named «MY_VAR». Virtual environment variables are specific to the virtual environment created by a tool like virtualenv, and are stored in the activate script for the virtual environment.

Accessing environment variables using Python

In Python, all environment variables are stored in the os.environ dictionary. The use of this conventional dictionary syntax is the simplest method for retrieving a variable from within your programme. You can use this method, for instance, to obtain the environment variable USER:

import os
user = os.environ['USER']

With this approach, Python will throw a KeyError exception if you attempt to import a nonexistent environment variable:

database_url = os.environ['DATABASE_URL']

Output:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/miguel/.pyenv/versions/3.8.6/lib/python3.8/os.py", line 675, in __getitem__
    raise KeyError(key) from None
KeyError: 'DATABASE_URL'

1. How to get an environment variable in Python?

The os.environ object in Python is used to retrieve environment variables. Although the os.environ object resembles a dictionary, it differs because only strings may be used as values, and they cannot be serialised to JSON. When it comes to referencing the os.environ object, you have a few choices:

a. Standard way

import os
os.environ['VAR_NAME']

b. Importing Environ Object

from os import environ
environ['VAR_NAME']

c. Renaming Environ to Env

from os import environ as env
env['VAR_NAME']

Depending on what should occur if an environment variable is missing, there are three different ways to access a given environment variable in Python.

2. How to set an environment variable in Python?

In Python, setting an environment variable is equivalent to setting a dictionary’s key:

os.environ['TESTING'] = 'true'

The only permitted values in os.environ are strings, which distinguishes it from a typical dictionary:

os.environ['TESTING'] = True

Output:

TypeError: str expected, not bool

Although there are use cases for setting them as well, your application will typically just need to get environment variables.

For example, constructing a DB_URL environment variable on application start-up using DB_HOST, DB_PORT, DB_USER, DB_PASSWORD, and DB_NAME environment variables:

os.environ['DB_URL'] = \
    'psql://{user}:{password}@{host}:{port}/{name}'.format(user=os.environ['DB_USER'
        ], password=os.environ['DB_PASSWORD'], host=os.environ['DB_HOST'
        ], port=os.environ['DB_PORT'], name=os.environ['DB_NAME'])

Look at another illustration of setting a default value of a variable following the value of another variable:

Set DEBUG and TESTING to ‘True’ if ENV is ‘development’.

if os.environ.get('ENV') == 'development':
    os.environ.setdefault('DEBUG', 'True') # Only set to True if DEBUG not set
    os.environ.setdefault('TESTING', 'True') # Only set to True if TESTING not set

3. How to delete an environment variable in Python?

Use the os.environ.pop function to remove a Python environment variable. For example, you might want to remove the other DB_ prefixed fields to make sure that the app can only access the database through the DB _URL that we previously gave.

Let’s see an additional example of removing an environment variable when it is no longer required:

auth_api(os.environ['API_KEY']) # Use API_KEY
os.environ.pop('API_KEY') # Delete API_KEY as it's no longer needed

4. How to retrieve list of Environment Variables?

To see every environment variable in Python, you could use the os.environ object. It denotes environment variables with their values. It provides a dictionary with the environmental variable names as the “keys” and their values as the “values”.

In the following programme, we loop through the dictionary returned by the os.environ. The key represents the name of the environment variables, and the value represents the value of the environment variables:

import os

for name, value in os.environ.items():
    print("{0}: {1}".format(name, value))

1. No default value

Get the environment variable directly if your app crashes when it is not set:

Example 1:

print(os.environ['HOME'])

Output:

'/home/ubuntu'

Example 2:

print(os.environ['DOES_NOT_EXIST']

Output:

Will raise a KeyError exception

For instance, the application shouldn’t be able to launch if a key environment variable is not declared and a default value, such as a database password, cannot be supplied. Instead of the typical KeyError exception (which doesn’t explain why your app didn’t launch), if you could catch the KeyError exception and display a helpful message instead, then:

import os
import sys

# Ensure all required environment variables are set
try:
    os.environ['API_KEY']
except KeyError:
    print '[error]: `API_KEY` environment variable required'
sys.exit(1)

2. With a default value

Using the os.environ.get method and passing the default value as the second parameter, you can retrieve a default value back if an environment variable is missing:

If HOSTNAME doesn’t exist, presume local development and return localhost:

print(os.environ.get('HOSTNAME', 'localhost')

None is returned if the variable doesn’t exist and you call os.environ.get without specifying a default value.

assert os.environ.get('NO_VAR_EXISTS') == None

3. Conditional logic if the value exists

Although you might want to know if an environment variable exists, you don’t necessarily need to know its value. For instance, If the DEBUG environment variable is set, your application may enter «Debug mode.»

You can confirm an environment variable’s existence by itself:

if 'DEBUG' in os.environ:
	print('[info]: app is running in debug mode')
if os.environ.get('DEBUG') == 'True':
	print('[info]: app is running in debug mode')

Setting Environment Variables in Python with .env file

The number of environmental variables increases along with the size and complexity of a programme. Most of the projects experience growing difficulties when using environment variables for application configuration and secrets because there is a lack of clear and consistent strategy for how to manage them, especially when deploying to multiple environments.

A simple and better solution is to use a .env file to include all of the variables for a particular environment and use a Python library such as python-dotenv to parse the .env file and populate the os.environ object.

Install the python-dotenv library after creating and activating a new virtual environment to follow along:

1. Create

python3 -m venv ~/.virtualenvs/doppler-tutorial

2. Activate

source ~/.virtualenvs/doppler-tutorial/bin/activate

3. Install dotenv package

pip install python-dotenv

Save the following to a file with the extension .env (notice that it uses the same syntax as setting a variable in the shell):

API_KEY="357A70FF-BFAA-4C6A-8289-9831DDFB2D3D"
HOSTNAME="0.0.0.0"
PORT="8080"

Next, save the subsequent code to dotenv-test.py.

4. Renaming os.environ to env

from os import environ as envfrom dotenv import load_dotenv

load_dotenv()

print('API_KEY:  {}'.format(env['API_KEY']))
print('HOSTNAME: {}'.format(env['HOSTNAME']))
print('PORT:     {}'.format(env['PORT']))

After that, launch dotenv-test.py to verify that the environment variables are populating:

# python3 dotenv-test.py

API_KEY:  357A70FF-BFAA-4C6A-8289-9831DDFB2D3D
HOSTNAME: 0.0.0.0
PORT:     8080

Use cases of Environment Variables

1. It is used to store configuration information and application secrets, which your running application can access as needed.
2. Enhance security procedures. It safeguards private data. When there are parts of your code that you don’t want others to see, like API tokens that give access to your accounts you only need to call the environment variables. If the API token is stored as an environment variable rather than directly typing it out in the file, users will be able to use your code without having to hardcode their API tokens.
3. The usage of environment variables also eliminates the need to update your source code if your secrets change.
4. Without making any code modifications, you can deploy your application in any environment and ensures that sensitive information, such as API keys, is not leaking into the source code.
5. When configuring your Python programme, environment variables are a wonderful method to do it without having to update the source code. Third-party API keys, network ports, database servers, and any other unique choices that your application may require to function properly are common configuration elements that are frequently supplied to applications using environment variables.
6. You can avoid manually updating your environment variables. For example, when running a script on someone else’s computer, you don’t need to manually change the «user» portion of the path; instead, you can use an environment variable that returns the user’s name. This means that when utilising a script on a different system, you no longer need to remember to modify that.
7. You can do development deployment or stage in deployment or even production deployment with environment variables. Environment variables will keep all the secrets safe.

Final thoughts

In this blog, we looked at how to use the Python dotenv package and an.env file to manage environment variables.

Environment variables play a vital role to isolate sensitive data from your application. They assist in keeping your app’s secrets safe and make it simple for you to switch between sets of secrets based on the circumstances. But taking care of them adds another duty to your plate.

Your client ids, hostnames, database names, port numbers, secrets, and other data are frequently stored in environment variables in the form of key-value pairs to keep them safe.

python-dotenv can be a very helpful tool for handling various environment variables, sensitive data and configuration choices in your Python applications and helps to avoid hard-coding sensitive information into your code.

Environment variables files are text files with the .env extension that can be used to store environmental variables. It’s a quick and easy approach to save global variables that you want to utilise but don’t want to be accessible to the public.

---

Atatus: Python Performance Monitoring

Atatus is an
Application Performance Management (APM) solution that collects all requests to your Python applications without requiring you to change your source code. However, the tool does more than just keep track of your application’s performance.

Monitor logs from all of your Python applications and systems into a centralized and easy-to-navigate user interface, allowing you to troubleshoot faster using
Python monitoring.

We give a cost-effective, scalable method to centralized Python logging, so you can obtain total insight across your complex architecture. To cut through the noise and focus on the key events that matter, you can search the logs by hostname, service, source, messages, and more. When you can correlate log events with
APM slow traces and errors, troubleshooting becomes easy.

Try your 14-day free trial of Atatus.

To set and get environment variables in Python you can just use the os module:

import os

# Set environment variables
os.environ['API_USER'] = 'username'
os.environ['API_PASSWORD'] = 'secret'

# Get environment variables
USER = os.getenv('API_USER')
PASSWORD = os.environ.get('API_PASSWORD')

# Getting non-existent keys
FOO = os.getenv('FOO') # None
BAR = os.environ.get('BAR') # None
BAZ = os.environ['BAZ'] # KeyError: key does not exist.

Note that using getenv() or the get() method on a dictionary key will return None if the key does not exist. However, in the example with BAZ, if you reference a key in a dictionary that does not exist it will raise a KeyError.

Environment variables are useful when you want to avoid hard-coding access credentials or other variables into code. For example, you may need to pass in API credentials for an email service provider in order to send email notifications but you wouldn’t want these credentials stored in your code repository. Or perhaps you need your code to function slightly differently between your development, staging and production environments. In this case you could pass in an environment variable to tell your application what environment it’s running in. These are typical use cases for environment variables.

Storing local env variables

You should write your Python code so that it is able to access environment variables from whatever environment it is running in. This could be either your local virtual environment that you’re using for development or a service that you are hosting it on. A useful package that simplifies this process is Python Decouple, this is how you would use it.

First install Python Decouple into your local Python environment.

$ pip install python-decouple

Once installed, create a .env file in the root of your project which you can then open up to add your environment variables.

$ touch .env   # create a new .env file
$ nano .env    # open the .env file in the nano text editor

‌You can then add your environment variables like this:

USER=alex
KEY=hfy92kadHgkk29fahjsu3j922v9sjwaucahf

Then save (WriteOut) the file and exit nano. Your environment variables are now stored in your .env file. If you’re using git, remember to add .env to your .gitignore file so that you don’t commit this file of secrets to your code repository.

Now that you have your environment variables stored in a .env file, you can access them in your Python code like this:

from decouple import config

API_USERNAME = config('USER')
API_KEY = config('KEY')

The benefit of using something like the above approach is that when you deploy your application to a cloud service, you can set your environment variables using whatever method or interface the provider has and your Python code should still be able to access them. Note that it is common convention to use capital letters for names of global constants in your code.

Most cloud service providers will have a CLI or web interface that lets you configure the environment variables for your staging or production environments. For guidance in these cases you ‘ll need to refer to their documentation on how to set environment variables when using their service.

---

Join the Able Developer Network

If you liked this post you might be interested in the Able developer network, a new place for developers to blog and find jobs.

---

---