Bash Advanced Variable Idioms for Case Sensitivity Management

Bash Advanced Variable Idioms for Case Sensitivity Management

Whenever we work with textual strings, sooner or later the issue of case comes up. Does a word need to be fully uppercase, fully lowercase, with a capitalized letter at the start of the word or sentence, and so on.

An idiom is a natural language expression of a simple programming task. For example, in the sleep 10 command (which will pause the terminal one is working in for ten seconds), the word sleep is a natural language expression of what is a time based coding construct, developed in the Bash GNU coreutils software package.

There are a number of special variable-bound idioms (i.e. suffixes which can be added to a variable name, indicating what we would like to do with a given variable), which can be used in Bash to more easily do these types of conversions on the fly instead of having to use for example the Sed Stream Editor with a Regular Expression to do the same.

If you are interested in using regular expressions, have a look at our Bash Regexps For Beginners With Examples Advanced Bash Regex With Examples articles!

This makes working with variables that need case modification, or if statement testing a whole lot easier and provides great flexibility. Idioms can be added directly inside the if statements and do not need to employ a subshell with sed.

While the syntax looks slightly complex to start with, once you learn a little mental support trick to remember the right keys, you will be well on your way to use these idioms in your next script or Bash one-liner script at the command line!

In this tutorial you will learn:

  • How to use the ^, ^^, , and ,, Bash variable suffix idioms
  • How to use a regular expression [] range idiom in combination with these
  • How to use the ^ and , idioms directly from within if statements
  • Detailed examples exemplifying the use of ^, ^^, , and ,,

Read more

String concatenation in Bash

String concatenation in Bash

This tutorial will explain the Bash string concatenation by using examples. When it comes to bash scripting or programming in general, the concatenation refers to joining two or more string together to produce single unified output. Using Bash shell and bash scripting the string concatenation can be achieved in number for ways.

In this tutorial you will learn:

  • How to concatenate strings using echo command
  • How to concatenate strings in loop
  • How to concatenate string with command output

Read more

Big Data Manipulation for Fun and Profit Part 3

Big Data Manipulation for Fun and Profit Part 3

There have been two previous article in this series, which you may want to read first if you have not read them yet; Big Data Manipulation for Fun and Profit Part 1 and Big Data Manipulation for Fun and Profit Part 2.

In this series, we discuss various ideas and practical approaches for handling big data, or more specifically handling, transforming, mangling, munging, parsing, wrangling, transforming and manipulating the data at the Linux command line.

This third article in the series will continue explore Bash tools which can help us when processing and manipulating text-based (or in some cases binary) big data. As mentioned in the previous articles, data transformation in general is an semi-endless topic as there are hundreds of tools for each particular text format. Remember that at times using Bash tools may not be the best solution, as an off-the-shelf tool may do a better job. That said, this series is specifically for all those (many) other times when no tool is available to get your data in the format of your choice.

Finally, if you want to learn more about why big data manipulation can be both fun and profitable… please read Part 1 first.

In this tutorial you will learn:

  • Additional big data wrangling / parsing / handling / manipulation / transformation techniques
  • What Bash tools are available to assist you, specifically for text based applications
  • Various examples, showing different methods and approaches

Read more

Useful Bash Command Line Tips and Tricks Examples - Part 6

Useful Bash Command Line Tips and Tricks Examples – Part 6

In this article we have a look at obtaining some basic hardware, system and operating system configuration information directly from the command prompt in your terminal. Having this information, and knowing where to find it, often helps when configuring a system, researching optimal hardware and software solutions, and generally being in control of the Linux system you own or manage.

We will focus mostly on every day information, the kind that is required on a regular basis. Going more in-depth often requires a few online searches to first define alignment better, and then often requires additional skills to interpret the results or output shown.

For example, we will not be looking at the output of vmstat, though we will discover how to find out what CPU is in your system without opening the hood. We will also look at some basic resource limit variables and surrounding concerns. The target audience for this article is thus beginner to medium advanced.

This article is part of the Useful Bash Command Line Tips and Tricks Series.

In this tutorial you will learn:

  • Useful Bash command line tips, tricks and methods
  • How to interact with the Bash command line in an advanced manner
  • How to sharpen your Bash skills overall and become a more proficient Bash user

Read more

vim-logo

Vim editor basics in Linux

Vim needs no presentations: it is probably the most loved text editor by system administrators, also due to the fact that it is a clone and an improvement of the original Vi, which is included by default in practically all Linux and Unix based operating systems. Vim can be quite intimidating at first, and it has a steep learning curve; learning how to use it, however, can really increase our productivity. In this article we will learn the Vim basics.

In this tutorial you will learn:

  • How to install Vim
  • Why Vim is a “modal” text editor and what are the Vim modes
  • How to tweak Vim look and feel using the ~/.vimrc configuration file

Read more

Bash command line tips and tricks

The Bash command line provides nearly limitless power when it comes to executing nearly anything you want to do. Whether it is processing a set of files, editing a set of documents, handling big data, managing a system or automating a routine, Bash can do it all.

This tutorial is sure to arm you with the tools and methods you need to become a much more proficient Bash user. Even already advanced users will likely pickup something new and exciting.

Read more

Useful Bash command line tips and tricks examples - Part 3

Useful Bash command line tips and tricks examples – Part 3

Bash is a varied shell interface with many programming options, and a rich instructional language. It is easy to miss Bash features and dynamics, so this series introduces a number of tips, tricks, examples and gotchas when it comes to using Bash. For the first two article in this series, please see our article Useful Bash command line tips and tricks examples part 2 and Useful Bash command line tips and tricks examples part 3.

In this tutorial series you will learn:

  • Useful Bash command line tips, tricks and methods
  • How to interact with the Bash command line in an advanced manner
  • How to sharpen your Bash skills overall and become a more proficient Bash user

Read more

Useful Bash Command Line Tips and Tricks Examples - Part 4

Useful Bash Command Line Tips and Tricks Examples – Part 4

Continuing our series on useful Bash command line tips and tricks, in today’s article, we will explore grepping only what you need, and start of with a primer on pwd and how to discover the path a script was started from.

In this tutorial you will learn:

  • Useful Bash command line tips, tricks and methods
  • How to interact with the Bash command line in an advanced manner
  • How to sharpen your Bash skills overall and become a more proficient Bash user

Read more

Useful Bash Command Line Tips and Tricks Examples - Part 5

Useful Bash Command Line Tips and Tricks Examples – Part 5

In this article, we will explore user input: for all those times you want to ask the user to ‘press enter to continue’, or to actually read a string of input and store it into a variable for later processing. We will also look at how to find manual pages for built-in commands which otherwise may not seem to be available.

In this tutorial you will learn:

  • Useful Bash command line tips, tricks and methods
  • How to interact with the Bash command line in an advanced manner
  • How to sharpen your Bash skills overall and become a more proficient Bash user

Read more

How to connect to an FTP server using Python

How to connect to an FTP server using Python

FTP (File Transfer Protocol) needs no presentations: it is among the most used file transfer methods between one or more clients and a server. By design it supports both anonymous access and authentication, but in its most basic form it doesn’t provide data encryption, that’s why it is often secured via TLS.

A lot of FTP client applications are available on Linux, as for example Filezilla (graphical) or lftp (command line). Sometimes, however, we may want to access an FTP server programmatically, perhaps to schedule file transfers. One easy way to do this is by using a programming language like Python. In this tutorial we will learn how to use the ftplib library to interact with an FTP server.

In this tutorial you will learn:

  • How to create an instance of the ftplib.FTP class
  • How to list files on a remote FTP server
  • How to upload files in binary and “lines” mode
  • How to download files in binary and “lines” mode
  • How to create,delete and rename directories and files
  • How to change working directory

Read more

Bash Background Process Management

Bash Background Process Management

There are many times when a Bash developer or user will want to run a process in the background, either from the command line or from inside a bash script, and then handle that same process again later. There are various command line tools which allow one to do so. Being able to start, manage and destroy background processes is a requirement for many more advanced level tasks, especially in the areas of advanced scripting and process control.

In this tutorial you will learn:

  • How to start, handle and/or manage, and destroy background processes
  • What command line tools are available to assist you with Bash process management
  • Examples highlighting the use of background processes at the Bash command line
Bash Background Process Management

Bash Background Process Management

Read more

Correct Variable Parsing and Quoting in Bash

Correct Variable Parsing and Quoting in Bash

Incorrect quoting in original source code can easily lead to bugs when the input provided by users is not as expected or not uniform. Over time, when Bash scripts change, an unforeseen side effect of an incorrectly quoted variable can lead to a bug even in otherwise untouched code. This is even more important for security related applications which may be prone to hacking attempts. Learn how to do quoting and variable parsing/validation properly from the outset, and avoid many of these issues! Let’s get started…

In this tutorial series you will learn:

  • How to quote your Bash variables properly
  • The caveats and results of incorrect quoting
  • How to ensure variable values are what they are supposed to be
  • How to check for empty, numeric and text based variable values
Correct Variable Parsing and Quoting in Bash

Correct Variable Parsing and Quoting in Bash

Read more