Python Libraries for DevOps

Reading JSON and YAML in Python

As a DevOps Engineer handling various file formats like JSON and YAML is crucial for managing configurations, data, and automation tasks. Python provides several libraries to work with these file formats efficiently. Here's a brief overview of how to read JSON and YAML files in Python using some commonly used libraries:

1. JSON Parsing: JSON (JavaScript Object Notation) is a widely used data interchange format. In Python, you can handle JSON files using the built-in json module.

import json

# Read JSON from a file
with open('data.json', 'r') as json_file:
    data = json.load(json_file)

# Now, 'data' contains the parsed JSON data as a Python dictionary or list
print(data)

1. YAML Parsing: YAML (YAML Ain't Markup Language) is another popular format for configuration files. To work with YAML files in Python, you can use the pyyaml library.

Install the library if you haven't already:

pip install pyyaml

Then, use it in your Python code:

import yaml

# Read YAML from a file
with open('data.yaml', 'r') as yaml_file:
    data = yaml.safe_load(yaml_file)

# Now, 'data' contains the parsed YAML data as a Python dictionary or list
print(data)

Other Python Libraries for DevOps tasks:

1. os and sys: These are built-in Python modules that allow you to interact with the operating system, handle file paths, execute system commands, and manage system-specific functionality.

2. requests: This library is used for making HTTP requests in Python, which is crucial for interacting with APIs, web services, and cloud platforms.

3. paramiko: A library for handling SSH connections and performing remote operations on servers, enabling automation in infrastructure management.

4. fabric: Built on top of paramiko, Fabric is a high-level library for streamlining the use of SSH for application deployment or server management.

5. subprocess: This module allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. It's useful for running external commands from Python.

6. boto3: For managing and interacting with Amazon Web Services (AWS) resources, this library is indispensable.

7. docker: If you work with containers, this library helps you interact with Docker from Python.

These are just some examples of libraries commonly used by DevOps Engineers in Python. Depending on your specific tasks, you may encounter many other libraries catering to different needs. Remember to install libraries not included in the standard Python distribution using pip before using them in your code.

Task-1: Create a Dictionary in Python and write it to a json File.

Creating a dictionary in Python and writing it to a JSON file is straightforward using the json module. Here's how you can do it:

import json

# Create a sample dictionary
data = {
    'name': 'Keshav',
    'age': 23,
    'email': 'keshav@example.com',
    'is_subscribed': True,
    'hobbies': ['reading', 'coding', 'traveling']
}

# Define the path to the JSON file
file_path = 'data.json'

# Write the dictionary to the JSON file
with open(file_path, 'w') as json_file:
    json.dump(data, json_file, indent=4)

print(f"Dictionary written to '{file_path}'.")

In this example, we create a dictionary called data with various keys representing personal information. We then use the json.dump() function to write this dictionary to a file called 'data.json' in a human-readable format with indentation.

After running this script, you'll find a 'data.json' file in the same directory with the content of the dictionary. The file will look like this:

{
    "name": "Keshav",
    "age": 23,
    "email": "keshav@example.com",
    "is_subscribed": true,
    "hobbies": [
        "reading",
        "coding",
        "traveling"
    ]
}

Now you have successfully created a dictionary and saved it to a JSON file in Python.

Task-2:

Read a json file services.json kept in this folder and print the service names of every cloud service provider.

To achieve the desired output, we need to read the JSON file and extract the service names for each cloud service provider. We can use the json module to load the JSON data and then access the service names for AWS, Azure, and GCP. Here's how you can do it:

import json

# Load JSON data from the file
with open('services.json', 'r') as json_file:
    data = json.load(json_file)

# Extract and print service names for each cloud service provider
print(f"AWS: {data['services']['aws']['name']}")
print(f"Azure: {data['services']['azure']['name']}")
print(f"GCP: {data['services']['gcp']['name']}")

{
    "services": {
      "debug": "on",
      "aws": {
        "name": "EC2",
        "type": "pay per hour",
        "instances": 500,
        "count": 500
      },
      "azure": {
        "name": "VM",
        "type": "pay per hour",
        "instances": 500,
        "count": 500
      },
      "gcp": {
        "name": "Compute Engine",
        "type": "pay per hour",
        "instances": 500,
        "count": 500
      }
    }
  }

Assuming the JSON data provided in the 'services.json' file is as you have given, the output will be:

AWS: EC2
Azure: VM
GCP: Compute Engine

This code loads the JSON data, accesses the relevant service names within the dictionary, and then prints them out for each cloud service provider. Make sure the 'services.json' file is in the same directory as the Python script, or you can provide the correct path to the file if it's located elsewhere.

Task-3 :

Read YAML file using python, file services.yaml and read the contents to convert yaml to json

To read the YAML file and convert it to JSON, you can use the pyyaml library to parse the YAML data and then use the json library to convert it to JSON. Here's how you can do it:

import yaml
import json

# Load YAML data from the file
with open('services.yaml', 'r') as yaml_file:
    yaml_data = yaml.safe_load(yaml_file)

# Convert YAML data to JSON
json_data = json.dumps(yaml_data,indent=2)

# Print the JSON data
print(json_data)

Assuming the 'services.yaml' file is in the same directory as the Python script, this code will read the YAML data, convert it to JSON, and then print the JSON representation of the YAML data. The output will look like:

{
    "services": {
        "debug": "on",
        "aws": {
            "name": "EC2",
            "type": "pay per hour",
            "instances": 500,
            "count": 500
        },
        "azure": {
            "name": "VM",
            "type": "pay per hour",
            "instances": 500,
            "count": 500
        },
        "gcp": {
            "name": "Compute Engine",
            "type": "pay per hour",
            "instances": 500,
            "count": 500
        }
    }
}

Now you have successfully read the YAML file and converted it to JSON using Python.

NOTE: before using yaml please install it using pip install pyyaml