FTP (File Transfer Protocol) needs no presentations: it is among the most used file transfer methods between one or more clients and a server. By design it supports both anonymous access and authentication, but in its most basic form it doesn’t provide data encryption, that’s why it is often secured via TLS.

A lot of FTP client applications are available on Linux, as for example Filezilla (graphical) or lftp (command line). Sometimes, however, we may want to access an FTP server programmatically, perhaps to schedule file transfers. One easy way to do this is by using a programming language like Python. In this tutorial we will learn how to use the ftplib library to interact with an FTP server.

In this tutorial you will learn:

  • How to create an instance of the ftplib.FTP class
  • How to list files on a remote FTP server
  • How to upload files in binary and “lines” mode
  • How to download files in binary and “lines” mode
  • How to create,delete and rename directories and files
  • How to change working directory

In this article, we will explore user input: for all those times you want to ask the user to ‘press enter to continue’, or to actually read a string of input and store it into a variable for later processing. We will also look at how to find manual pages for built-in commands which otherwise may not seem to be available.

In this tutorial you will learn:

  • Useful Bash command line tips, tricks and methods
  • How to interact with the Bash command line in an advanced manner
  • How to sharpen your Bash skills overall and become a more proficient Bash user

Continuing our series on useful Bash command line tips and tricks, in today’s article, we will explore grepping only what you need, and start of with a primer on pwd and how to discover the path a script was started from.

In this tutorial you will learn:

  • Useful Bash command line tips, tricks and methods
  • How to interact with the Bash command line in an advanced manner
  • How to sharpen your Bash skills overall and become a more proficient Bash user

Incorrect quoting in original source code can easily lead to bugs when the input provided by users is not as expected or not uniform. Over time, when Bash scripts change, an unforeseen side effect of an incorrectly quoted variable can lead to a bug even in otherwise untouched code. This is even more important for security related applications which may be prone to hacking attempts. Learn how to do quoting and variable parsing/validation properly from the outset, and avoid many of these issues! Let’s get started…

In this tutorial series you will learn:

  • How to quote your Bash variables properly
  • The caveats and results of incorrect quoting
  • How to ensure variable values are what they are supposed to be
  • How to check for empty, numeric and text based variable values
Correct Variable Parsing and Quoting in Bash
Correct Variable Parsing and Quoting in Bash

There are many times when a Bash developer or user will want to run a process in the background, either from the command line or from inside a bash script, and then handle that same process again later. There are various command line tools which allow one to do so. Being able to start, manage and destroy background processes is a requirement for many more advanced level tasks, especially in the areas of advanced scripting and process control.

In this tutorial you will learn:

  • How to start, handle and/or manage, and destroy background processes
  • What command line tools are available to assist you with Bash process management
  • Examples highlighting the use of background processes at the Bash command line
Bash Background Process Management
Bash Background Process Management

Whether you are an IT professional who needs to download 2000 online bug reports into a flat text file and parse them to see which ones need attention, or a mum who wants to download 20 recipes from an public domain website, you can benefit from knowing the tools which help you download webpages into a text based file. If you are interested in learning more about how to parse the pages you download, you can have a look at our Big Data Manipulation for Fun and Profit Part 1 article.

In this tutorial you will learn:

  • How to retrieve/download webpages using wget, curl and lynx
  • What the main differences between the wget, curl and lynx tools are
  • Examples showing how to use wget, curl and lynx
Retrieving Webpages Using wget, curl and lynx
Retrieving Webpages Using wget, curl and lynx

In the first part of this big data manipulation series - which you may want to read first if you haven’t read it yet; Big Data Manipulation for Fun and Profit Part 1 - we discussed at some length the various terminologies and some of the ideas surrounding big data, or more specifically as it relates to handling, transforming, mangling, munging, parsing, wrangling, transforming and manipulating the data. Often these terms are use interchangeably and often their use overlaps. We also looked at the first set of Bash tools which may help us with work related to these terms.

This article will explore a further set of Bash tools which can help us when processing and manipulating text-based (or in some cases binary) big data. As mentioned in the previous article, data transformation in general is an semi-endless topic as there are hundreds of tools for each particular text format. Remember that at times using Bash tools may not be the best solution, as an off-the-shelf tool may do a better job. That said, this series is specifically for all those (many) other times when no tool is available to get your data in the format of your choice.

And, if you want to learn why big data manipulation can be both profitable and fun… please read Part 1 first.

In this tutorial you will learn:

  • More big data wrangling / parsing / handling / manipulation / transformation techniques
  • What Bash tools are available to help you, specifically for text based applications
  • Examples showing different methods and approaches
Big Data Manipulation for Fun and Profit Part 2
Big Data Manipulation for Fun and Profit Part 2

These days everyone seems to be speaking about Big Data - but what does it really mean? The term is used quite ambiguously in a variety of situations. For the purposes of this article, and the series, we will refer to big data whenever we mean ‘a large amount of textual data, in any format (for example plain ASCII text, XML, HTML, or any other human-readable or semi-human-readable format). Some techniques shown may work well for binary data also, when used with care and knowledge.

So, why fun (ref title)?

Handling gigabytes of raw textual data in a quick and efficient script, or even using a one-liner command (see Linux Complex Bash One Liner Examples to learn more about one-liners in general), can be quite fun, especially when you get things to work well and are able to automate things. We can never learn enough about how to handle big data; the next challenging text parse will always be around the corner.

And, why profit?

Many of the world’s data is stored in large textual flat files. For example, did you know you can download the full Wikipedia database? The problem is that often this data is formatted in some other format like HTML, XML or JSON, or even proprietary data formats! How do you get it from one system to another? Knowing how to parse big data, and parse it well, puts all the power at your fingertips to change data from one format to another. Simple? Often the answer is ‘No’, and thus it helps if you know what you are doing. Straightforward? Idem. Profitable? Regularly, yes, especially if you become good at handling and using big data.

Handling big data is also referred to as ‘data wrangling’. I started working with big data over 17 years ago, so hopefully there is a thing or two you can pickup from this series. In general, data transformation as a topic is semi-endless (hundreds of third-party tools are available for each particular text format), but I will focus on one specific aspect which applies to textual data parsing; using the Bash command line to parse any type of data. At times, this may not be the best solution (i.e. a pre-created tool may do a better job), but this series is specifically for all those (many) other times when no tool is available to get your data ‘just right’.

In this tutorial you will learn:

Big Data Manipulation for Fun and Profit Part 1
Big Data Manipulation for Fun and Profit Part 1

PHP developers and web server admins can use the phpinfo function to quickly see information about their installation of PHP. This can assist in debugging, seeing what version of PHP is installed, or seeing various configuration options.

On Linux systems, it's common to make a phpinfo.php page after installing a LAMP server or LEMP server to make sure that PHP is working and to verify settings.

In this tutorial, we'll guide you through the creation of a phpinfo.php page on your own system, as well as how to access this file afterwards.

In this tutorial you will learn:
  • How to create phpinfo.php page

Bash one-liners can reduce workload, automate something quickly and put the power of ultimate system control in your hands. Over time, you will likely learn to write more complex one-liners and some of the things you end up writing as a seasoned professional will be nearly in-parsible by a beginner. That said, the Bash command and development language is highly structured - and relatively easy to understand - once you know about the in and outs. It really is like becoming proficient in a foreign language.

In this tutorial you will learn:

  • How to write more advanced Bash one-liner commands and scripts
  • Understand how to combine various commands into one-liner scripts
  • Understand how exit codes from one command can affect other commands when using && and ||
  • Understand how input from a command can be modified and then be used by the next command
  • Usage and real-life like examples of more advanced Bash one-liners

Bash is a varied shell interface with many programming options, and a rich instructional language. It is easy to miss Bash features and dynamics, so this series introduces a number of tips, tricks, examples and gotchas when it comes to using Bash. For the first two article in this series, please see our article Useful Bash command line tips and tricks examples part 2 and Useful Bash command line tips and tricks examples part 3.

In this tutorial series you will learn:

  • Useful Bash command line tips, tricks and methods
  • How to interact with the Bash command line in an advanced manner
  • How to sharpen your Bash skills overall and become a more proficient Bash user

If you read our previous linux subshells for beginners with examples article, or are experienced with subshells already, you know that subshells are a powerful way to manipulate Bash commands inline, and in a context sensitive manner.

In this tutorial you will learn:

  • How to create more advanced subshell commands
  • Where you can employ more advanced subshells in your own code
  • Examples of more advanced subshell commands

Making use of subshells in Bash provides you with an ability to generate context sensitive information from right within your Bash command. For example, if you want to modify a text string right inside an echo statement, then this can be done easily with subshells.

In this tutorial you will learn:

  • How to use employ the use of subshells in Bash
  • How to use subshells to obtain context sensitive information
  • Basic Bash subshell usage examples

In this series we are exploring various tips, tricks and Bash command line examples which will help you become a more advanced Bash user and coder. Bash provides a rich scripting and coding language which puts the power back in the hands of the user and developer. Bash also allows you to learn as you go along, thereby making it a more enjoyable experience. For the first article in our series, please see our article Useful Bash command line tips and tricks examples part 1.

In this tutorial series you will learn:

  • Useful Bash command line tips, tricks and methods
  • How to interact with the Bash command line in an advanced manner
  • How to sharpen your Bash skills overall and become a more proficient Bash user

You may already be versed in debugging Bash scripts (see How to Debug Bash Scripts if you are not familiar with debugging Bash yet), yet how to debug C or C++? Let’s explore.

GDB is a long-standing and comprehensive Linux debugging utility, which would take many years to learn if you wanted to know the tool well. However, even for beginners, the tool can be very powerful and useful when it comes to debugging C or C++.

For example, if you’re a QA engineer and would like to debug a C program and binary your team is working on and it crashes, you can use GDB to obtain a backtrace (a stack list of functions called - like a tree - which eventually led to the crash). Or, if you are a C or C++ developer and you just introduced a bug into your code, then you can use GDB to debug variables, code and more! Let’s dive in!

In this tutorial you will learn:

  • How to install and use the GDB utility from the command line in Bash
  • How to do basic GDB debugging using the GDB console and prompt
  • Learn more about the detailed output GDB produces

FIND LATEST LINUX JOBS on LinuxCareers.com
Submit your RESUME, create a JOB ALERT or subscribe to RSS feed.
LINUX CAREER NEWSLETTER
Subscribe to NEWSLETTER and receive latest news, jobs, career advice and tutorials.
DO YOU NEED ADDITIONAL HELP?
Get extra help by visiting our LINUX FORUM or simply use comments below.

You may also be interested in: