Sunday, June 3, 2018

Configuring OpenVPN on AWS EC2

This tutorial focus on configuring OpenVPN on Amazon AWS EC2. We will not be going through the process of creating an Amazon account. For details about Amazon AWS please refer to the main site https://aws.amazon.com/.

Acknowledgement

We would like to shout-out to Paul Bischoff for his post on "How to make your own free VPN with Amazon Web Services" He has recently updated the post for easy-rsa configuration. He made it easy for installing and configuring easy-rsa. However, we did not follow his openvpn configuration. For configuration we refer to openvpn HOWTO articles.


Create an Instance (Create a Linux Server)

First login to Amazon AWS, then navigate to EC2 Dashboard.  Select Services and EC2.



Once we are in the EC2 we should have the screen below



On the left column select EC2 Dashboard.


We should have the screen similar to below.


Please also note that we can create server in any region we choose. To switch region, please look out for the location at the top right corner next to our userid. Click on the location and select the region we want. Similar EC2 Dashboard will appear.

At EC2 Dashboard, scroll down the main page to display the part on "Create Instance"


Click on Launch Instance. The screen will show various server image for us to choose. See screenshot below:



Select the first image by clicking on Select. The screen will show the details of our selected image. Please note that for new user Amazon provide a 12 month free tier if we do not exceed certain usage limit. The first image is eligible for the free tier.



The screen above show us the CPU capacity and RAM allotment for our server. We can select other server and pay for the usage, for a start, we will stick to the free tier. Please scroll down to configure our security group.


Under Security group name. Enter the security group name. We use mySecurity as the group name. Next click Add Rule and configure the rules as follows:

Type:SSH, Protocol:TCP, Port:22, Source:Anywhere
Type:Custom TCP, Protocol:TCP, Port:943, Source:Anywhere
Type:Custom TCP, Protocol:TCP, Port:443, Source:Anywhere
Type:Custom UDP Protocol:UDP, Port:1194, Source:Anywhere

See screenshot below:


Please note that there will be warning since our source is anywhere. If we want to be able to access the server from anywhere, we should leave the source as anywhere. Click Review & Launch. In the next step, we review all the settings such as image and security group.



Once we are satisfy with the settings, scroll to the end and click Launch.


The next step is to create key pair. Please note that for EC2, we do not login to the server with our Amazon account. We will not be using any account to login the server. We connect to the server using key pair.



Select Create a new key pair and enter the key pair name. Once it is done, click to Download Key Pair. Once the key is with us, please safe keep this key file. we would not be able to login to the server with the key file.

Scroll down and click Launch Instances


The system will response with the launch status as shown below:




Managing Instances

This section briefly discuss, the very basic of managing instance. Please note that if we want to remain in the free tier, there are limits on how many instances we can create. 

First, select Services on the top left corner and select EC2 as shown below:


Once we are in the EC2 we should have the screen below



On the left column select EC2 Dashboard.

EC2 Dashboard is similar to the screen below:



Now, Click on Instances. We should have the screen below:


This instances summary tell us what instances we have are running. Next, we would like to pause and explain a bit about EC2 instances. 

Once important information we need is the IP address. Once an instance is launch, it will be assigned with a public address. We need the ip address to connect to the server.


Under normal tier, every instances contributed to the billing meter. We can reboot the instances. Please note that the billing meter will still be counting when you reboot.

If we stop the instances, the data will not be lost provided we use EBS-backed instances. For normal instance without EBS-backed instance, data will be lost. Only the ip address remains.

However, if we terminate the instances, the billing meter stop. Please note this important fact. Once we terminated the instances, the data, configuration and ip address for the server will be lost.

To manage the instance, select the instance we want and click on Actions:

Under Instance State, we can Stop, Reboot or Terminate.

If we select Stop, the system will warn us the all data will be lost.


If we select Terminate, the system will warn us that all will be lost


Let refrain from stopping or terminating the instance. In the next section, we will be connecting to the instance/server.

Connecting to Instance

In this section, we will be showing how to connect to the server via ssh. Please note that this section is for Mac/Linux user. For Windows user, please refer to the article mentioned at the beginning.

First we copy the file myServerKey.pem to ~/.ssh folder. If we do not have we can just copy to the home drive. After we start a SSH session, a hidden folder will be created in the home drive.

First, open a terminal. 

The follwoing command is to copy the session key to the .ssh folder.
cp ~/Downloads/myServerKey.pem ~/.ssh/

If there is no .ssh folder, use the home folder first. Before the next session, we can copy the file to .ssh folder.
cp ~/Downloads/myServerKey.pem ~

Next, we need to change the permission:
chmod 400 myServerKey.pem

We connect to ssh with:
ssh -i ~/.ssh/myServerKey.pem ec2-user@xxx.xxx.xxx.xxx 
(xxx refers to ip address from our instance summary)

Once the connection is successful, we have the following screen



Next, we would like to prepare the server for openvpn.

Installing OpenVPN and Preparation

Before we start installing openvpn, we need to update the system as follows:
# EC2 maintenance
sudo yum update -y

Next, we install openvpn and easy-rsa
# Install openvpn 
sudo yum install openvpn -y
sudo yum install easy-rsa -y --enablerepo=epel

Next, we would like to create a directory for easy-rsa under /etc/openvpn. This way any configuration will not be lost when there is an update. It is not advisable to store the configuration under /usr/share folder.
# make a direcotry for easy-rsa
sudo mkdir /etc/openvpn/easy-rsa
cd /etc/openvpn/easy-rsa
sudo cp -rv /usr/share/easy-rsa/3.0.3/* ./


Setup OpenVPN and Generate Keys

To setup openvpn, first we initialize PKI and build the certificate authority:
# Build PKI and CA
sudo ./easyrsa init-pki
sudo ./easyrsa build-ca
Please note that we will be asked to create the password for the CA. It is advisable to create a good and long password.

Next we generate generate a Diffie-Hellman key. This is to provide forward secrecy.
sudo ./easyrsa gen-dh

Next we generate the vpn server certificate. Please create a good password for the server certificate. Please note that for the second command, it will prompt a signing password. We need to use CA PASSWORD FOR SIGNING.
# Generate server cert and signed
# Create a good server password
sudo ./easyrsa gen-req server
# When prompt, use CA password for signing
sudo ./easyrsa sign-req server server

Next, we generate client certificate. Similarly, create a good password for the client certificate. If we are generating the client certificate for family or friends; we may need to ask them to enter the passphrase. Similarly, use CA password for signing certificate.
# Generate client
sudo ./easyrsa gen-req client
sudo ./easyrsa sign-req client client

We can create as many client certificate according to our needs.

Finally for added security we add TLS security by generating a ta.key. This feature is to prevent DDOS attack.
# add TLS security
cd /etc/openvpn
sudo openvpn --genkey --secret ta.key



Copy Keys

We need to copy keys generated to client for connection. For that, we prefer to create a separate folder and park the necessary keys to the folder.

Please note that while preparing the key files, it is easier if we use operate as root.

# *********************************************
# Below is the step to copy key files to a folder for scp

sudo su
cd /etc/openvpn
mkdir keys
cp ta.key keys
cp /etc/openvpn/easy-rsa/pki/dh.pem keys
cp /etc/openvpn/easy-rsa/pki/ca.crt keys
cp /etc/openvpn/easy-rsa/pki/private/ca.key keys
cp /etc/openvpn/easy-rsa/pki/private/client.key keys
cp /etc/openvpn/easy-rsa/pki/issued/client.crt keys
cd keys
chmod 777 *

# ls to confirm
# exit from root
exit
# ************************************************

Please note that all the keys and certificate are necessary for the client to use except one. The key that are not required is ca.key. We copy this key for keep offline since it is not necessary for the server to use this key.

Just a reminder .crt files are public key and .key are private key. If we are working with multiple clients, then we can only give them the appropriate client private key. Clients can have all the crt files. ta.key are for extra security, similar to dh.pem.

#################################################################################
# Below is instruction for local machine
# Copy from local machine
scp -i ~/.ssh/myServerKey.pem ec2-user@xxx.xxx.xxx.xxx:/etc/openvpn/keys/* ~/localpath/
#################################################################################

Finally, we clean up the ca.key and change the permission to the more restrictive one.
# Must only do after ca.key is copied to local machine
sudo rm /etc/openvpn/easy-rsa/pki/private/ca.key
sudo rm /etc/openvpn/keys/ca.key
cd /etc/openvpn/keys
sudo chmod 600 *

Summary we only provide client with the follow:
  • ta.key
  • dh.pem
  • ca.crt
  • client.crt - must be the same certificate create for the user.
  • client.key - must be the same key create for the user.
Please also reminded to safe keep the ca.key.

Setup OpenVPN Server Configuration

We can get a sample server configuration from the openvpn site. Please copy the conf file to /etc/openvpn and use any favorite editor to edit the configuration.

Listed below are the configuration we use:

We add 2 lines as shown below
askpass
auth-nocache

The following are changes we made or setting we uncomment:
# SSL/TLS root certificate (ca), certificate
ca /etc/openvpn/easy-rsa/pki/ca.crt
cert /etc/openvpn/easy-rsa/pki/issued/server.crt
key /etc/openvpn/easy-rsa/pki/private/server.key

# Diffie hellman parameters.
dh /etc/openvpn/easy-rsa/pki/dh.pem

push "redirect-gateway def1 bypass-dhcp"

push "dhcp-option DNS 8.8.8.8"
push "dhcp-option DNS 8.8.4.4"

# Enable compression on the VPN link and push the
compress lz4-v2
push "compress lz4-v2"

# You can uncomment this out on
# non-Windows systems.
user nobody
group nobody

The follow are the defaults we use:
port 1194
proto udp
dev tun
server 10.8.0.0 255.255.255.0
ifconfig-pool-persist ipp.txt
keepalive 10 120
tls-auth ta.key 0 # This file is secret
cipher AES-256-CBC
persist-key
persist-tun
status openvpn-status.log
verb 3
explicit-exit-notify 1


Start OpenVPN Server

To start openvpn use the command
sudo service openvpn start

To check the log please refer to /var/logs/messages


Configuring Client

Before we start connecting the vpn session, we need to configure the client file. Copy the sample file from openvpn site.

We add the following line:
auth-nocache

We only made changes to the server ipaddress:
# The hostname/IP and port of the server.
remote ip-address-my-server-1 1194

The following are default:
client
dev tun
proto udp
resolv-retry infinite
nobind
persist-key
persist-tun
#Need to change if keys are in different location
ca ca.crt
cert client.crt
key client.key
remote-cert-tls server
tls-auth ta.key 1
cipher AES-256-CBC
verb 3

For Mac OS X user, we do not recommend the client software provide by openvpn, instead we recommend to use Tunnelblick. We can get their software at https://tunnelblick.net/. Please follow the site tutorial on how to use tunnelblick. But it is quite intuitive.

For Ubuntu (Debian class) or Red Hat class user,  we can install openvpn using yum or apt-get. After that we can launch the session using the command line

# To connect openvpn vis command line
sudo openvpn --config '/pathToClientConfigFile'
enter client password when asked.

Alternatively, we can also use network manager.

Connecting From Client and Troubleshooting

Now, we can start the session. To test connection, please ping 10.8.0.1. If ping test is successful then we are good to go.

If client failed to make connection, first check the server log to see if openvpn is running.

If server is running well with no error and client still failed to make connection, perhaps we should try to make connection in a public wifi. Try to make connection from different location to see if the problem lies with the firewall.

If connection is successful but we cannot use the browser, then we need to check the DNS settings. Please remember to check the server configuration file.

Update 15 Jun 18

Some Open WiFI Network Do Not Support UDP

When you have established vpn connection on a wifi network, but you could not browse the internet. There are some troubleshooting steps involved.

The first issue is name resolution, so you need to ping the DNS server. If DNS server is reachable, we should clear our DNS cache. Please also check wether we could browse using other browser.

If all ease is fine, there is one possibility that some public wifi disable udp port. In this case we need to reconfigure our server to run on TCP port.

Please note that using TCP port for openvpn will slow down the transmission speed.

For server configuration, we just need to change the proto from udp to tcp. Please also make sure to add a custom TCP rule that open port 1194 in the AWS security group. Very IMPORTANT, please comment out explicit-exit-notify as it is not supported in tcp.

Please also remember to change the client from proto udp to proto tcp



***



Wednesday, May 23, 2018

Local Machine, Virtual Machine and Cloud

Previously, when we need to install Linux on a local machine, we need to get extra hard disk space, and we need to configure grub so that our PC will dual boot. If we need to install more Linux distribution we need to partition more hard disk space. That happened in the last decade.

When virtual machine mature, we are able to install VMWare, VirtualBox, or Parallel in our host system. From the virtual machine application, we can create many disk image where each image contain different operating system. Using virtual machine also has its restriction. First, we need a lot of hard disk space to keep the image; second, we need lost of RAM if we were to run multiple virtual machine (vm) at the same time. Furthermore, we need to bring external hard disk to run the vm for mobility.

Now, there is a better way, although it might cost a bit of money. We would like to introduce AWS, Google Cloud and Azure. Although there products are more suitable for big organization. This does not prevent individual user from utilizing these features. In fact it save cost a lot of cost for small business.

Going forward we will be publish more articles on the cloud platform.

***




Thursday, November 2, 2017

What is Machine Learning?

A machine learning algorithm is a computer program that is able to perform a task without being explicitly programmed to do so. Machine learning programs are trained to perform a certain task or make some prediction based on historical examples. The successful completion of the task or correct prediction will be measured as the performance metric. The learning program should be able to improve its performance metric with increase in historical example.

Supervised Learning

In supervised learning, we are given a dataset (training data) with expected result and we training the computer program using some algorithm based on the training data. 

Supervised learning can be further classified into regression problem such as prediction housing prices, stock price and production volume etc. Supervised learning can also be classified into classification problem where the task of the computer program is to classified a problem into 2 or more categories. For example we can classified a tumour as malignant for benign. 

In supervised learning, first we use predetermined dataset as training data. Then we choose and adopt a learning algorithm before we start training the learning algorithm with our data.

There are many machine learning algorithm. The most basic ML algorithm is Linear Regression. However, we do not apply linear regression very often since most things we want to predict are not in a linear form. Most commonly, we used polynomial regression algorithm. 

We use Logistic Regression for classification problem. Besides Linear/Polynomial Regression and Logistic Regression, we also use Neural Networks and Support Vector Machine for supervised learning.

Unsupervised Learning

For unsupervised learning, we feed the computer program with raw data without expected or predetermined result. The program suppose to examine the raw data and define its structure by detecting similar features or characteristics.

For unsupervised learning, we use K-Means clustering and PCA (Principal Component Analysis). We also use Gaussian Abnormally Detection Algorithm to monitor and detects abnormality.


*** End ***


Sunday, September 14, 2014

Quickly Boot to Bootcamp Windows

If you are using Bootcamp, you will notice that you can boot to Max OS X quickly using the system tray icon. In order to boot to Windows you need to go to preference and select the startup disk. Is there a way to boot into Windows quickly? Yes.

There are several solutions including using the startup disk under system preference and installing an app for it. I will show you how to boot to Windows using Startup disk. Then I will show you the instructions to create an shortcut or alias so that access the startup disk quickly.

Using Startup Disk

1. Select System Preference.


2. Click Startup Disk


3. Select "BOOTCAMP Windows" and click restart.

Create a Shortcut to Startup Disk

1. Open a Finder. Goto the first level folder of your disk.


2. Select System >> Library >> PreferencePanes. If you prefer to use command line the path is /System/Library/PreferencePanes


3. Under PreferencePanes, find StartupDisk.prefPane. Create a shortcut or alias by pressing Command+Option and drag the icon to the desktop.


4. You can also add StartupDisk.PrefPane to the dock by dragging the icon from your desktop to the dock as shown below. Please note that you can only place the icon next to the folders not application. 


Now once you click the icon from the desktop or dock, you are given the screen below immediately.



*****

Gmail Server Rejects Password from Email Client and Mobile Apps

If you encountered problem when login into gmail account (using IMAP) from your email clients such as Thunderbird, Live Mail, Apple Mail, Mail apps in iPhone and iPad, and any other email clients or apps; you are not alone. The problem you encountered is not a bug but it is a security feature implemented by Google. The solution is as follows:

Problems

When you add a gmail account in any email client using IMAP, you might have an error that says "your id and password has been rejected by the server". Another situation occurs when you previously working gmail account suddenly stops working and it produce a similar message that you id and password has been rejected by the mail server.

Possible Causes

1. Google Security Feature
Google has turn on a feature that prevents less secure apps and devices from logging into the server. This might includes emails clients and mobile apps.


Under the section Less secure apps, access for less secure apps is disabled. This is the more secure option. However, you might not be able to use your other email client such as Apple Mail, Windows Live Mail, Thunderbird, Mail apps from iPhone and iPad and mobile apps. You can still use your gmail apps in iOS and Android.

If you clicked enabled for "Access for less secure apps", you allows email clients and mobile apps to access. This is the less secure option. You can use any other non Google products to access your email.

2. IMAP Not Enabled
IMAP is disabled by default. If you have been accessing your email by web or Google apps, you might not notice that IMAP is disabled by default.


3. Your Email Access is Locked
After multiple tries, you access to the email server is being locked. Even if enabled the previous two options, you might still encounter problems because you have not unlocked your account. To unlock the email access you need to go to this site:





Solutions (Step by Step)

  1. Remove any configuration from you emails clients and mobile apps.
  2. Goto https://accounts.google.com/DisplayUnlockCaptcha 
  3. Click "Continue
  4. Login to the email site using web mail.
  5. Click the drop arrow next to your login id (on the top right corner) and click Accounts
  6. Select the tabs "Security" 
  7. Find the box under "Accounts Permission" 
  8. Click Settings.  You will see the following.
  9. Click Enable. Click "Done".
  10. Go to gmail site, click "Settings" (i.e. the gear) and select settings. 
  11. Click "Forwarding and POP/IMAP" as shown. 
  12. Check the IMAP settings. Click "Enable IMAP" and Click "Save Changes" at the bottom.
  13. Make sure that IMAP is enabled. 
  14. Configure your email client or mobile apps again. It should be working. If it is still not working, repeat step 2 and 3. Check to make sure that all settings are changed and wait for a while before you configure you email client again.

That's it and good luck.

*****