How to count occurrence of a specific character in a string or file using bash

Below you can find some hints on how to count an occurrence of specific character in a file or in a string. Le’s say we have a string “Hello Bash”:

$ STRING="Hello Bash"
$ echo $STRING
Hello Bash

Using bash shell we can now count an occurrence of any given character. For example let’s count number of occurrences of a character l:

$ echo $STRING | sed -e 's/\(.\)/\1\n/g' | grep l | wc -l
2

Read more

BASH Scripting: Parenthesis Explained

Author: Tobin Harding
Here we briefly outline some of the major use cases for brackets, parenthesis,
and braces in BASH scripting, see bottom of page for definition of
these three terms.

Double parentheses (( )) are used for arithmetic:

((var++))
((var = 3))
for ((i = 0; i < VAL; i++))
echo $((var + 2))

Read more

How to test for null or empty variables within Bash script

The following bash script example we show some of the way how to check for an empty or null variable using bash:

#!/bin/bash 

if [ -z "$1" ]; then
    echo "Empty Variable 1"
fi


if [ -n "$1" ]; then
    echo "Not Empty Variable 2"
fi


if [ ! "$1" ]; then
    echo "Empty Variable 3"
fi


if [ "$1" ]; then
    echo "Not Empty Variable 4"
fi


[[ -z "$1" ]] && echo "Empty Variable 5" || echo "Not empty Variable 5"

Read more

SyntaxError: Non-ASCII character – Python with UTF-8 encoding

Question

:
My Python program produce a following error message upon execution:

 SyntaxError: Non-ASCII character '\xc4' in file test.py on line 1, but no encoding declared; 

Answer:

Normally the above error message is displayed by python when other characters other then ASCII are used withing your code. The solution is to either remove all non-ASCII characters or include the bellow line into your code to enable UTF-8 encoding:

# - *- coding: utf- 8 - *-

Read more

How to obtain an user input with Python raw_input function example

The Python raw_input() function is used to read a string from standard input such as keyboard. This way a programmer is able to include user inserted data into a program. Let’s start with a simple example using python script to ask for an user name.

print "What is your name?"
name = raw_input()
print "Hello %s!" % name

First, we print string What is your name? telling the user what we expect him to input. Next, using the raw_input() function the standard input is assigned to a variable name. Lastly, we print the value of variable name to standard output.

$ python input.py 
What is your name?
Monty Python
Hello Monty Python!

Read more

How to access and print command line arguments with Python

The following is an example on how to pass and access command line arguments which a Python script. Save the following python script to a file eg. python-arguments.py

from sys import argv

name, first, second, third, fourth = argv

print "Script name is:", name
print "Your first argument is:", first
print "Your second argument is:", second
print "Your third argument is:", third
print "Your fourth argument is:", fourth


# Alternatively we can access "argv" argument list directly using range. For exmaple:

# Print all arguments except script name
print argv[1:]

# Print second argument
print argv[2]

# Print second and third argument
print argv[2:4]

# Print last argument
print argv[-1]

Read more

R programming software environment Docker image deployment and usage

About

The automated build Docker image of The R Project for Statistical Computing “linuxconfig/cran-r” can be used to instantly deploy R programming software environment on any hosts given that you have docker already installed on your system.

Configuration

The docker image with the R Project is compiled and runs on Debian GNU/Linux system.

Usage

Here we assume that docker is already installed on your system.First pull docker image:

# docker pull linuxconfig/cran-r

Read more

Extract all URLs using Beautiful Soup and Python3

The following link will extract all URL’s for a given web page.

#!/usr/bin/env python3

# Python Version:  3.4.2
# bs4 version: 4.3.2-2

from urllib.request import urlopen
from bs4 import BeautifulSoup

html = urlopen("http://gnu.org") # Insert your URL to extract
bsObj = BeautifulSoup(html.read());

for link in bsObj.find_all('a'):
    print(link.get('href'))

Read more

An overview of GNU R programming language

Introduction

The aim of this article is to provide an overview of the GNU R programming language. It starts a series of articles devoted to programming with R. Its objective is to present, in an organized and concise manner, the elementary components of the R programming language. It is designed to help you understand R code and write your own. It is assumed that the reader has already some basic programming knowledge of R. If you are not familiar with any of R features it is recommended that you first read A quick GNU R tutorial to basic operations, functions and data structures.

Expressions

An R expression is an elementary component of R code. Expression in R can be:

  • assignment statement;
  • conditional statement;
  • arithmetic expression.

Examples of R expressions:

> y<-100
> if (1==1) 1 else 0
[1] 1
> 100/5
[1] 20

R expression are constructed from objects or functions. It is common to separate them with a new line, however, you can also separate expressions with semicolons as below.

Read more

How to Make a Basic Intrusion Detection System with Bash

Introduction

For most of us WEP encryption has become a joke. WPA is quickly going the same way thanks to many tools such as Aircrack-ng. On top of this, wired networks are no strangers to unwanted guests as well. Anyone serious about security should have a good Intrusion Detection system in their toolbox.

There are already some very good IDS’s (Intrusion Detection Systems) available. Why would anyone want to re-invent the wheel in Bash??? There are a couple of reasons for this. Obviously Bash scripts can be very light weight. Especially compared to some of the GUI programs that are out there. While programs like Etherape suck us in with pretty colors, they require constant monitoring to know when the network has changed. If you are like most of us, you only use the computer for two things, work and play. By using the system bell to alert for new clients online you can leave this script running and not have to have a constant watch. If you do decide you want to inspect what a suspicious client is doing more closely, you can always open up etherape, wireshark, or your tool of choice. But until you have a problem you can play or work on other things.

Another bonus to this program is that it will only show ip addresses on the networks connected to your computer. If you were hosting a busy server or perhaps downloading the latest Linux distro though a torrent client, an IDS may be flooded with connections. Looking for a new malicious client can be like looking for a needle in a hay stack. While this script may seem simple compared to other IDS’s, simplicity can have its perks too.

What you will need

Nmap is required for this script to work. We will not be doing any port scanning. However, to make this script fast we needed something better than a regular ping. Nmap’s -sP parameter will only use a ping scan to check if a clients up. There were some variations in how Nmap outputs information between versions. So far this script has only been tested using Nmap 5.00 (Debian Squeeze) and 5.21 (Debian Sid). You may have luck with other distros and versions of Nmap. However, with all the possibilities I could only support a couple at this time.

Read more

Linux Command Line & Bash Shell Shortcuts

Introduction

Although you may think that you have learned to master Linux command line with bash shell, there are always some new tricks to learn to make your command line skills more efficient. This article will teach you a few more basic tricks on how to make your life with the Linux command line & bash more bearable and even enjoyable.

Bash Command History Expansion

This section will mostly deal with bash shortcuts in combination with three bash history expansion characters “!”, “^” and “#”. Bash Command History Expansion character “!” indicates start of history expansion. The “^” is a substitution character to modify a previously run command. The last optional character is “#”, which denotes the reminder of the line as a comment.

Repeat last command

$ echo Bash Shortcuts
Bash Shortcuts
$ !!
echo Bash Shortcuts
Bash Shortcuts

!! is probably the easiest and most popular bash shortcut, which simply shows and executes your last entered command.

Read more