Diagnosing LAN Speeds

After having network issues/degradation while trying to access a work server, I had to diagnose the network the server is connected to. I had to set myself on a mission – and after realising that the seems to be very limited tools for such things, I stumbled upon ‘iperf‘.

Iperf is a command-line tool used in the diagnostics of network speed issues, it measures the maximum network throughput a server can handle. It is particularly useful when experiencing network speed issues, as you can use Iperf to determine which server is unable to reach maximum throughput.

Iperf installation

Iperf must be installed on both computers you are testing the connection between.

sudo apt-get install -y iperf

TCP Clients & Servers

Iperf requires two systems because one system must act as a server, while the other acts as a client. The client connects to the server you’re testing the speed of.

  1. On the node you wish to test, launch Iperf in server mode:
iperf -s

You should see output similar to:

Screenshot_2017-05-05_12-02-15

2. On your second Linode, connect to the first. Replace dbelab04 with the first node’s IP address in my case i’m using the hostname

iperf -c dbelab04

You should see output similar to:
Screenshot_2017-05-05_12-02-53

3. You will also see the connection and results on your Iperf server. This will look similar to:

Screenshot_2017-05-05_12-03-04

4. To stop the Iperf server process, press CTRL + c.

————————————————————————————————–

You can do pretty much the same thing with plain old nc (netcat) if you’re that way inclined. On the server machine:

nc -vvlnp 12345 >/dev/null

You should see something similar to:

Screenshot_2017-05-05_12-14-05

And the client can pipe a 10Gb of zeros through dd over the nc tunnel.

dd if=/dev/zero bs=10M count=1K status=progress | nc -vvn 192.168.4.23 12345

You should see something similar to:

Screenshot_2017-05-05_12-13-54

The timing there is given by dd but it should be accurate enough as it can only output as fast the pipe will take it. If you’re unhappy with that you could wrap the whole thing up in a time call.

Remember that the result is in megabytes so multiply it by 8 to get a megabits-per-second speed. The demo above is running at 11.8mbps due to my laptops network limitation and number of hops…

Python’s Virtualenv working with Jenkins

If you use virtualenv to isolate your python project’s environment, and want your code tested automatically — read on, else ignore.

virtualenv isolates your project’s python environment

virtualenv makes sure you lock down your project’s main directory and all subdirectories of it. This ‘lockdown’ means that you never touches your global python binary, or any globally installed libraries (like “sudo pip install ipython” ).

Once locked down, you install all packages again, even those you have globally installed. This enables you to have one version of flask globally installed, but another version in your project. All dependencies can be listed in a separate file and validate a precise environment for you to work with. Tightly controlled dependencies is key to a deployment without surprises.

Jenkins checks the health of your project for each change

Jenkins is a CI server which means it does a lot of repeating stuff so you can focus on doing more important stuff. More specifically, it listens for changes to your project’s version control system (like git).

When changes are detected, the project is built and the test suite is executed. If any step fails, the CI server tells you that it did.

Setup Jenkins, and make it use virtualenv

Jenkins needs some massaging before it handles the hijacked environment of virtualenv. This is how I did it for my local git repository:

  • Download and install Jenkins
  • Start it, it should be up on http://localhost:8080
  • Install the Git Plugin
  • Setup a new project with these properties:
    • Source Code Management: add the URI to your local repository,
    • /Users/you/Sites/asdf in my case. Make sure the jenkins user can read this directory, otherwise the Jenkins GUI will tell you something random about invalid git repo, without a hint about a permissions error.
    • Build Triggers: Poll SCM (with an interval like 0 * * * *). This is needed because
      you’re too lazy to build manually; and
      you can not trigger builds with a git post-commit hook otherwise
    • Build > Execute shell. I’ve used two steps, one for setting up the environment and one for the actual tests:

# Setup a proper path, I call my virtualenv dir "venv" and
# I've got the virtualenv command installed in /usr/local/bin
PATH=$WORKSPACE/venv/bin:/usr/local/bin:$PATH
if [ ! -d "venv" ]; then
        virtualenv venv
fi
. venv/bin/activate
pip install -r requirements.txt --download-cache=/tmp/$JOB_NAME

and


. venv/bin/activate
which python

# reply should be
/home/user/.virtualenv/bin/python

Setting up an MQTT server on Debian Jessie

Jolabs Tech Blog

Intro

Today we`re going to be setting up our own home automation server in a dedicated linux server. It’s gonna host our platform using the the Node.Js environment. The different devices around the house are going to be speaking MQTT to this server. I’ve chosen MQTT because of it’s robustness and lightweightness allowing for easy deployment on my favourite controllers: ESP8266. The esp-01, still the cheapest to this date,only sports 4mb of flash, has no trouble at all controlling a couple of relay’s for now. In the future the newer/bigger boards can be used to get more IO and function, but for now I’m going to be using the esp-01’s extensively around the house flexibly (it’s wireless…).

Requirements

The idea is to have the server online 24/7, so consider a computer thats not to power-hungry; like a Raspberry Pi or a small embedded box computer. Alternatively you can boot up…

View original post 340 more words

Node-Red Dashboard running on Nginx port 80

After finishing my Node-Red dashboard app, and boy, it’s awesome. Everything works great, but I wanted to bind it to port 80 on my nginx server.

Nginx

A more popular approach is to set up Nginx as a reverse proxy by having it bind to the desired port, forwarding all incoming traffic to my node red dashboard.

Nginx is a high performance, open source web server (similar to Apache) that is widely-used.

The main benefit of Nginx is the fact that it takes care of transport optimization.

Installing Nginx

sudo apt-get install nginx
 

Configuring Nginx

Next, we’ll need to configure Nginx so that it forwards traffic to my dashboard. Let’s start off by removing the default configuration file:

sudo rm /etc/nginx/sites-enabled/default
 

Next, create a new file in /etc/nginx/sites-available/ called node and open it with nano:

sudo nano /etc/nginx/sites-available/node
 

Paste the following code in the file and make sure to change example.com to your domain (or IP), and 1880 to your Node-Red Dashboard port:

server {
    listen 80;
    server_name example.com;

    location /ui {
        proxy_set_header   X-Forwarded-For $remote_addr;
        proxy_set_header   Host $http_host;
        proxy_pass         "http://127.0.0.1:1880";
    }
}

The proxy_pass declaration configures Nginx to act as a reverse proxy by forwarding all incoming requests on port 80 to Node-Red dashboard on port 1880, on behalf of the client.

Next, we need to symlink our configuration to sites-enabled for it to be used by Nginx, since it’s currently in sites-available:

sudo ln -s /etc/nginx/sites-available/node /etc/nginx/sites-enabled/node

Applying the Configuration

Let’s restart Nginx so that it loads our configuration:

sudo service nginx restart

run

sudo service nginx status

and you should see something like:

* nginx.service - A high performance web server and a reverse proxy server
   Loaded: loaded (/lib/systemd/system/nginx.service; enabled)
   Active: active (running) since Sat 2017-02-11 14:14:14 UTC; 306ms ago
  Process: 3103 ExecStop=/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /run/nginx.pid (code=exited, status=0/SUCCESS)
  Process: 3111 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
  Process: 3108 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
 Main PID: 3113 (nginx)
   CGroup: /system.slice/nginx.service
           |-3113 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
           |-3114 nginx: worker process
           |-3115 nginx: worker process
           |-3116 nginx: worker process
           `-3119 nginx: worker process

Feb 11 14:14:14 alexapi nginx[3108]: nginx: [warn] server name "127.0.0.1:1880/ui" has suspicious symbols in /etc/nginx/sites-enabled/node:3
Feb 11 14:14:14 alexapi nginx[3111]: nginx: [warn] server name "127.0.0.1:1880/ui" has suspicious symbols in /etc/nginx/sites-enabled/node:3
Feb 11 14:14:14 alexapi systemd[1]: Started A high performance web server and a reverse proxy server.

All set! Nginx will now forward all incoming requests to your app and even survive a server crash, since it automatically starts up with your machine.

Screenshot_2017-02-11_16-18-49.png

also see: https://www.digitalocean.com/community/tutorials/how-to-connect-your-internet-of-things-with-node-red-on-ubuntu-16-04

 

If you found this helpful, Please donate $10 towards my sister’s LLB Degree fees, click on link below for more info.

https://www.feenix.org/student/dc8eb44488524a7f8c9635fd7b3600ff-2017

Thank you in advance.

Raspberry Pi – Stuck in boot after installing upstart

I  installed upstart so I could run a script at startup [native /etc/init/*.conf]  with sudo apt-get install upstart and I then rebooted my RPi.

Afterwards my RPi running Debian-Jessie was pretty much unresponsive via the network. Right after connecting it to my monitor I then realised that it was stuck on boot.

CODE:

[   17.777252] init: samba-ad-dc main process (513) terminated with status 1
[  133.770712] init: failsafe main process (364_ killed by TERM signal
[info] Using makefile-style concurrent boot in runlevel S
...
[info] Loading kernel module fuse.

Only at that moment, I started realising that I should have probably backed up my SD Card before, but then it was already too late. It then struck me – So I will have to redo it all over again with a new and fresh raspbian installation. That was not something I was looking forward to do ,especially not after I have invested my time to this project I have undertaken myself on[apts, pips and nodes].

And then it hit me, A while ago – I had more or less the same issue on my Xubuntu 12.04 install and managed to get away with it via chroot which then worked, then started researching as to if it is possible to chroot my ARM Raspbian install from an Wbuntu system. To my revelation it was…And below is how I managed to rollback to Systemd-sysv without having to burn a new raspbian image to my sd card.

First, install two packages on your Ubuntu system: qemu-user, and proot.

sudo apt-get install -y qemu-user proot

After you mount the Raspbian SD card, you can do the equivalent of a ‘chroot‘ with:

 sudo proot -q qemu-arm  -r /mnt/raspbian_sd_card

I was then able to chroot/proot to my raspbian install and removed upstart:

sudo apt-get -y purge --auto-remove upstart

This command will also automatically re-install systemd-sysv

System V init script(start-stop-daemon)

I’ve been looking for efficient ways to start at boot my NodeJS dependent applications, with inspiration from https://gist.github.com/alobato/1968852, I modified it to my own needs.

Link might interest you as well:
http://big-elephants.com/2013-01/writing-your-own-init-scripts/
https://www.cyberciti.biz/tips/linux-write-sys-v-init-script-to-start-stop-service.html

Copy template to /etc/init.d and rename it to something meaningful. Then edit the script and enter that name after Provides:(between ### BEGIN INIT INFO and ### END INIT INFO).

#!/bin/bash
# Inspired by https://gist.github.com/alobato/1968852
# Needs Provides, Descriptions

### BEGIN INIT INFO
# Provides:
# Required-Start: $all
# Required-Stop: $all
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description:
# Description:
### END INIT INFO

set -e
NAME=""
PIDFILE="/run/$NAME/$NAME.pid"
# Application one wants to upstart
DAEMON=""
DAEMON_OPTS=""
# Run as user
RUN_USER=""
RUN_GROUP=""
function daemon_run {
	mkdir -p /run/$NAME
	chown $RUN_USER:$RUN_GROUP /run/$NAME
	start-stop-daemon --start --background --quiet --chuid $RUN_USER:$RUN_GROUP --chdir /run/$NAME --pidfile $PIDFILE --make-pidfile --exec $DAEMON $DAEMON_OPTS
}
exec > /var/log/$NAME.log 2>&1

case "$1" in
start)
	echo -n "Starting $NAME ... "
	daemon_run
	echo "done."
;;

silent)
	echo -n "Starting $NAME in silent mode ... "
	daemon_run
	echo "done."
;;

stop)
	echo -n "Stopping $NAME ... "
	start-stop-daemon --stop --quiet --oknodo --pidfile $PIDFILE --remove-pidfile
	echo "done."
;;
restart|force-reload)
	echo -n "Restarting $NAME ... "
	start-stop-daemon --stop --quiet --oknodo --retry 30 --pidfile $PIDFILE --remove-pidfile
	alexa_run
	echo "done."
;;
*)
	echo "Usage: $0 {start|stop|restart}"
exit 1
esac
exit 0

When done,

sudo systemctl enable 'name_of_filename'
reboot

Mpho