Anaconda is a distribution of python and other open source packages that are meant to be used for scientific computing. It is frequently used for data science, predictive analytics, and machine learning. Installing Anaconda is the fastest way to have all of the tools for scientific computing readily available to you. It includes the conda package manager, IPython the interactive python shell, the spyder IDE, along with the Project Jupyter interactive web based computational environments: Jupyter Notebook, and JupyterLab.
python
How to launch external processes with Python and the subprocess module
In our automation scripts we often need to launch and monitor external programs to accomplish our desired tasks. When working with Python, we can use the subprocess module to perform said operations. This module is part of the programming language standard library. In this tutorial we will take a quick look at it, and we will learn the basics of its usage.
In this tutorial you will learn:
- How to use the “run” function to spawn an external process
- How to capture a process standard output and standard error
- How to check the exist status of a process and raise an exception if it fails
- How to execute a process into an intermediary shell
- How to set a timeout for a process
- How to use the Popen class directly to pipe two processes
How to read and create csv files using Python
CSV is the acronym of “Comma Separated Values”. A csv file is a just plain text document used to represent and exchange tabular data. Each row in a csv file represents an “entity”, and each column represents an attribute of it. Columns are usually separated by a comma but other characters can be used as field separator instead of it. In this tutorial we will see how to read and create csv files using Python and specifically the csv module, which is part of the
language standard library.
In this tutorial you will learn:
- How to read csv rows as a list of strings
- How to read a csv as a list of dictionaries
- How to create a csv using Python
- How to create a csv starting from a list of dictionaries
How to connect to an FTP server using Python
FTP (File Transfer Protocol) needs no presentations: it is among the most used file transfer methods between one or more clients and a server. By design it supports both anonymous access and authentication, but in its most basic form it doesn’t provide data encryption, that’s why it is often secured via TLS.
A lot of FTP client applications are available on Linux, as for example Filezilla (graphical) or lftp (command line). Sometimes, however, we may want to access an FTP server programmatically, perhaps to schedule file transfers. One easy way to do this is by using a programming language like Python. In this tutorial we will learn how to use the ftplib library to interact with an FTP server.
In this tutorial you will learn:
- How to create an instance of the ftplib.FTP class
- How to list files on a remote FTP server
- How to upload files in binary and “lines” mode
- How to download files in binary and “lines” mode
- How to create,delete and rename directories and files
- How to change working directory
Install pip on Linux
pip is the package manager for the Python coding language. It can be installed on a Linux system and then used on the command line to download and install Python packages and their requisite dependencies.
This gives developers – as well as users who are just executing Python programs but not developing them – an easy way to download software packages written in Python. It’s available for installation on any major Linux distro and operates in much the same way as a distro’s package manager, which you’re probably already familiar with.
In this guide, we’ll show you how to install pip for Python 2 and Python 3 on various Linux distributions. We’ll also show you basic usage commands for pip, such as installing and removing software packages.
In this tutorial you will learn:
- How to install pip for Python 2 and Python 3 on major Linux distros
- Basic usage commands for pip
Python Regular Expressions with Examples
A regular expression (often abbreviated to “regex”) is a technique, and a textual pattern, which defines how one wants to search or modify a given string. Regular expressions are commonly used in Bash shell scripts and in Python code, as well as in various other programming languages.
In this tutorial you will learn:
- How to start with Regular Expressions on Python
- How to import regex Python module
- How to match strings and characters using Regex notation
- How to use the most common Python Regex notations
How to perform HTTP requests with python – Part 2 – The request Library
In the previous article we saw how to perform basic HTTP requests using the python3 standard library. When requests become more complex, or we just want to use less code, and we don’t mind adding a dependency to our project, it’s possible (and sometimes even recommended) to use the external requests
module. The library, which adopted the “HTTP for Humans” motto, will be the focus of this article.
In this tutorial you will learn:
- How to perform HTTP requests with python3 and the ‘requests’ library
- How to manage server responses
- How to work with sessions
How to parse a json file from Linux command line using jq
The JSON
(JavaScript Object Notation) format is widely used to represent data structures, and is frequently used to exchange data between different layers of an application, or by the use of API calls. We probably know how to interact with json-formatted data with the most used programming languages such as parsing JSON with python, but what if we need to interact with it from the command line, or in a bash script? In this article we will see how we can accomplish such a task by using the jq
utility and we will learn its basic usage.
In this tutorial you will learn:
- How to install jq in the most used Linux distributions or compile it from source
- How to use jq to parse json-formatted data
- How to combine filters using “,” and “|”
- How to use the length, keys, has and map functions
How to create and manipulate tar archives using Python
On Linux and other Unix-like operating systems, tar is undoubtedly one of the most used archiving utilities; it let us create archives, often called “tarballs”, we can use for source code distribution or backup purposes. In this tutorial we will see how to read, create and modify tar archives with python, using the tarfile
module.
In this tutorial you will learn:
- The modes in which a tar archive can be opened using the tarfile module
- What are the TarInfo and TarFile classes and what they represent
- How to list the content of a tar archive
- How to extract the content of a tar archive
- How to add files to a tar archive
Introduction to python web scraping and the Beautiful Soup library
Objective
Learning how to extract information out of an html page using python and the Beautiful Soup library.
Requirements
- Understanding of the basics of python and object oriented programming
How to perform HTTP requests with python – Part 1: The standard Library
HTTP is the protocol used by the World Wide Web, that’s why being able to interact with it programmatically is essential: scraping a web page, communicating with a service APIs, or even simply downloading a file, are all tasks based on this interaction. Python makes such operations very easy: some useful functions are already provided in the standard library, and for more complex tasks it’s possible (and even recommended) to use the external requests
module. In this first article of the series we will focus on the built-in modules. We will use python3 and mostly work inside the python interactive shell: the needed libraries will be imported only once to avoid repetitions.
In this tutorial you will learn:
- How to perform HTTP requests with python3 and the urllib.request library
- How to work with server responses
- How to download a file using the urlopen or the urlretrieve functions
How to install PyCharm on Ubuntu 18.04 Bionic Beaver Linux
Objective
The objective is to install PyCharm community edition on Ubuntu 18.04 Bionic Beaver Linux
Operating System and Software Versions
- Operating System: – Ubuntu 18.04 Bionic Beaver
Requirements
Privileged access to your Ubuntu System as root or via sudo
command is required.