For the impatient : the full script is available on GitHub.  For all the others, let’s understand the problem and the proposed solution.

Why automating DNS configuration ?

This is a common problem with public cloud machines : at every start, the machine receives a different public IP address and public DNS name.  There are two methods to keep a consistent name to access your machine :

• bundle a Dynamic DNS client (such as inadyn) and access your machine through its DynDNS domain name.
• create a DNS A (address) or CNAME record (an alias) to point to the public IP address of your machine

The latter solution is only valid if you have your own domain name.  It offers the maximum flexibility as you can configure the DNS as you need.

To automatize the task, your DNS provider must provide you with a programmatic way to change its configuration : an API. This is exactly what Amazon’s Route 53 DNS Service offers you.

To complete my previous article, I choose to add at the end of my script a command to dynamically associate the instance public IP name to my own domain name, such as myservice.aws.stormacq.com.  I choose to define an alias to the public DNS name setup by AWS, using a CNAME record.

How to programmatically configure your Route 53 DNS ?

AWS’ Route 53 API is RESTful, making it easy to manipulate it with command line, using “curl” command for example.  curl can be used to issue GET requests to read configuration data and POST requests to change DNS configuration.

Requests payload is defined in XML.  An example GET query would be

<?xml version="1.0"?>
<ListHostedZonesResponse xmlns="https://route53.amazonaws.com/doc/2012-12-12/">
  <HostedZones>
    <HostedZone>
      <Id>/hostedzone/MY_ZONE_ID</Id>
      <Name>aws.mydomain.com.</Name>
      <CallerReference>22F684C6-3886-3FFF-8437-E22C5DCB56E7</CallerReference>
      <Config>
        <Comment>AWS Route53 Hosted subdomain</Comment>
      </Config>
      <ResourceRecordSetCount>4</ResourceRecordSetCount>
    </HostedZone>
  </HostedZones>
  <IsTruncated>false</IsTruncated>
  <MaxItems>100</MaxItems>
</ListHostedZonesResponse>

To restrict access to  your DNS configuration, the API requires authentication.  Route 53 mandate the use of a proprietary HTTP header to authenticate requests.

Full details about Route 53 API is available on Amazon’s documentation.

The problem when using authentication and curl

AWS’ Route 53 authentication is described with great details and examples in the official documentation. Basically, it is based on a HMAC signature computed from the current date/time and your AWS Secret Key.

The HTTP header to be added to the request is as following

AWS3-HTTPS AWSAccessKeyId=MyAccessKey,Algorithm=ALGORITHM,Signature=Base64( Algorithm((ValueOfDateHeader), SigningKey) )

The Algorithm can be HMacSHA1 or HMacSHA256. The date is the current system time.  You can use your system time or you can ask AWS what is their system time.  The latter needs an additional HTTP call but this method will avoid time synchronisation issues between your machine and AWS. While curl is very versatile and can accommodate to many different situations, it can not compute an HMac signature to send as authentication header, a short python script is my solution.

The Python Solution

I choose to wrap the curl call into Python, let Python compute the signature, generate the appropriate HTTP header and then call curl, passing all remaining command line arguments to curl itself. The general idea is as following :

• collect AWS_ACCESS_KEY and AWS_SECRET_KEY
• Compute the Signature
• Call curl with correct parameters to inject the authentication HTTP header and all command line parameters we have received

Signature

The signature is generated with this code.  It receives two String as input (the text to sign and the key). I hard-coded the algorithm. The function returns the base64 encoded signature.

def getSignatureAsBase64(text, key):
    import hmac, hashlib, base64
    hm  = hmac.new(bytes(key, "ascii"), bytes(text, "utf-8"), hashlib.sha256)
    return base64.b64encode(hm.digest()).decode('utf-8')

AWS’s date

Retrieving AWS’s date is similarly easy

def getAmazonDateTime():
    import urllib.request
    httpResponse=urllib.request.urlopen("https://route53.amazonaws.com/date")
    httpHeaders=httpResponse.info()
    return httpHeaders['Date']

Formatting the header

And the header is formatted with

def getAmazonV3AuthHeader(accessKey, signature):
    # AWS3-HTTPS AWSAccessKeyId=MyAccessKey,Algorithm=ALGORITHM,Signature=Base64( Algorithm((ValueOfDateHeader), SigningKey) )
    return "AWS3-HTTPS AWSAccessKeyId=%s,Algorithm=HmacSHA256,Signature=%s" % (accessKey,signature)

Calling curl

Finally, we just have to call the curl command :

        import subprocess
        curlCmd = ["/usr/bin/curl",
                        "-v" if DEBUG else "",
                        "-s", "-S",
                        "--header",
                        "X-Amzn-Authorization: %s" % AWS_AUTH,
                        "--header",
                        "x-amz-date: %s" % AWS_DATE]
        curlCmd += args.curl_parameters
        curlCmd += [args.curl_url]
        logging.debug(" ".join(curlCmd))                
        return subprocess.call(curlCmd)

The full script is available under a BSD license on GitHub.  There is some additional plumbery to handle command line arguments, to load the AWS credentials etc … which is out of the scope of this article.

Conclusion

Using this script, you can easy use curl command to GET or POST REST requests to Route 53′s API.

I am using this script to create custom CNAME records whenever an EC2 instance is started, allowing me to reuse a well known, stable DNS public name to access my instance. A sample XML to define a CNAME is posted on GitHub together with the source code.

Enjoy !

If you are totally new to EC2, I strongly advise you to follow a Getting Started guide before going through this article.

The VPN server I am using for the purpose of this article is based on IPSec / L2TP security protocols implemented by open source projects OpenSWAN and XL2LTP.

For the impatient, the scripts are available on github, along with basic configuration and setup information.  Should you need more details, I encourage you to read the following.

Why a private VPN server ?

Sometime, it is legitimate to create an encrypted tunnel of data to another machine on the internet. Think about situations like

• Being connected on a public network in an hotel, a conference, a restaurant or coffee shop
• Willing to escape your ISP or Service Provider limitations (Belgium DNS Blocking, French Hadopi, …)
• Accessing services not being distributed in your country (Deezer, Spotify etc ..)
• or simply to ensure no one can snoop your network traffic

How to start a customised machine on EC2 ? Some Background.

AWS provides several ways to start customised machines. Either you can create your own virtual machine image (AMI) based on one of the many images available.  Either you can start a standard image and run a script at startup time to customise it. Either you can boot from an EBS backed machine image (AMI) and create a snapshot of your root volume.

The first method is more labor intensive (install the software, maintain the image, …) and more expensive (you have to pay for the storage of your customised image) but has the advantage of faster startup times as the image does not need to install and to configure required softwares at boot time.

Running a script at boot time is easier as you do not need to enter into the details of creating and maintaining custom images.  It is cheaper as you do no need to store that custom image. But the machine is slower to boot as it requires to download, install and configure required softwares at every boot.  This is the method I choose to setup the VPN server.

The latter method (EBS Snapshot of root volume) is described in extenso in the documentation and – based on my own experience – provides the best ratio between labour, price and effectiveness.  This is probably the method I would recommend for production workloads.

But … How to start a customisation / installation script just after booting a standard linux distribution or one of the prepared Amazon Machine Image ?  This is where cloud-init kicks in.

Cloud-Init is an open source library initiated by Canonical (the maker Ubuntu) to initialise Virtual Machines running in the cloud. It allows, amongst others, to do post-boot configuration like

• setting the correct locale
• setting the hostname
• initialising (or installing) ssh keys
• setup mount points
• etc …

It also allows to pass a user defined script to the instance to perform any additional setup and configuration tasks.  This is the technique I am using to download, install, configure and start IPSec and L2TP daemons on the server.

Cloud-Init is included by default in Ubuntu machine images and in Amazon Linux machine images on EC2.

For the purpose of this article, I choose to use the Amazon Linux machine image because it is lightweight and specifically designed to run on EC2.

This is enough background information, let’s start to do real stuffs.

How to start a machine from your command line ?

To start an EC2 instance form your machine command line, you will need the following :

• an Amazon Web Service account and a credit card
• to create a SSH key pair
• to create a VPN security group

The VPN Security Group must allow TCP and UPD port 500 and UPD port 4500 as shown on the screenshot below.

VPN Security Group

Please refer to the getting started guide to learn how to perform these. Be sure to write down the name of your key pair and the name of your security group as we will need these later.

Once your basic setup of EC2 is done, you will need to install and configure EC2 Command line tools on your machine.

• Download and Install EC2 Command Line Tools
• Configure your environment

To configure your environment, you will need to setup a couple of environment variables, typically in $HOME/.profile

• EC2_HOME environment variable points to command line tools
• EC2_URL environment variables contains AWS endpoint (http://docs.aws.amazon.com/general/latest/gr/rande.html#ec2_region)
• AWS_ACCESS_KEY environment variable contains your AWS access key
• AWS_SECRET_KEY environment variable contains your AWS secret key

Do not change the name of these environment variables as these are used in the script.

For example, here is my own .profile file (on Mac OS X) :

export JAVA_HOME=`/usr/libexec/java_home`
export EC2_HOME=/Users/sst/Projects/aws/ec2-api-tools-latest
export AWS_ACCESS_KEY=<access key>
export AWS_SECRET_KEY=<secret key>
export EC2_URL=http://ec2.eu-west-1.amazonaws.com

Once this setup is done, you can start to use the EC2 command line tools as demonstrated in the script below :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

#to be run on my laptop
# create and start an instance
#AMI = AMZN Linux 64 Bits
#AMI_DESCRIPTION="amazon/amzn-ami-pv-2012.09.0.x86_64-ebs"
AMI_ID=ami-c37474b7
KEY_ID=sst-ec2
SEC_ID=VPN
BOOTSTRAP_SCRIPT=vpn-ec2-install.sh
 
echo "Starting Instance..."
INSTANCE_DETAILS=`$EC2_HOME/bin/ec2-run-instances $AMI_ID -k $KEY_ID -t t1.micro -g $SEC_ID -f $BOOTSTRAP_SCRIPT | grep INSTANCE`
echo $INSTANCE_DETAILS
 
AVAILABILITY_ZONE=`echo $INSTANCE_DETAILS | awk '{print $9}'`
INSTANCE_ID=`echo $INSTANCE_DETAILS | awk '{print $2}'`
echo $INSTANCE_ID > $HOME/vpn-ec2.id
 
# wait for instance to be started
DNS_NAME=`$EC2_HOME/bin/ec2-describe-instances --filter "image-id=$AMI_ID" --filter "instance-state-name=running" | grep INSTANCE | awk '{print $4}'`
 
while [ -z "$DNS_NAME" ]
do
 echo "Waiting for instance to start...."
 sleep 5
 DNS_NAME=`$EC2_HOME/bin/ec2-describe-instances --filter "image-id=$AMI_ID" --filter "instance-state-name=running" | grep INSTANCE | awk '{print $4}'`
done
 
echo "Instance started"
 
echo "Instance ID = " $INSTANCE_ID
echo "DNS = " $DNS_NAME " in availability zone " $AVAILABILITY_ZONE

You will need to slightly customise this script to make it run :

• Line 5 : check what is the AMI ID in your geography
• Line 6 : replace “sst-ec2″ with the name of your ssh key pair
• Line 7 : replace “VPN” with the name you choose for your Security Group

How is it working ?

10
11
12

echo "Starting Instance..."
INSTANCE_DETAILS=`$EC2_HOME/bin/ec2-run-instances $AMI_ID -k $KEY_ID -t t1.micro -g $SEC_ID -f $BOOTSTRAP_SCRIPT | grep INSTANCE`
echo $INSTANCE_DETAILS

This script starts an EC2 instance (line 11) of the given type with the specified SSH key pair and Security Group.  It uses the “-f” option to pass a cloud-init user data script that will download install and configure IPSec and L2TP once the machine is booted.

18
19
20
21
22
23
24
25
26

# wait for instance to be started
DNS_NAME=`$EC2_HOME/bin/ec2-describe-instances --filter "image-id=$AMI_ID" --filter "instance-state-name=running" | grep INSTANCE | awk '{print $4}'`
 
while [ -z "$DNS_NAME" ]
do
 echo "Waiting for instance to start...."
 sleep 5
 DNS_NAME=`$EC2_HOME/bin/ec2-describe-instances --filter "image-id=$AMI_ID" --filter "instance-state-name=running" | grep INSTANCE | awk '{print $4}'`
done

The script then waits for the machine to be ready (lines 19-26) and, once available, the script reports the machine public DNS name (to be used to configure your VPN client software) (line 30 – 31)

How to Install and to Configure VPN into your new machine ?

Now that the machine is started, it receives the customisation script through the -f option.  Cloud-Init will execute this script to finalise the setup of the machine.

Here is the script allowing to install and configure IPSec and L2TP automatically.  Some details are given after the code.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110

#!/bin/sh
 
# Please define your own values for those variables
 IPSEC_PSK=SharedSecret
 VPN_USER=username
 VPN_PASSWORD=password
 
# Those two variables will be found automatically
 PRIVATE_IP=`wget -q -O - 'http://instance-data/latest/meta-data/local-ipv4'`
 PUBLIC_IP=`wget -q -O - 'http://instance-data/latest/meta-data/public-ipv4'`
 
yum install -y --enablerepo=epel openswan xl2tpd
 
cat > /etc/ipsec.conf <<EOF
 version 2.0
 
config setup
 dumpdir=/var/run/pluto/
 nat_traversal=yes
 virtual_private=%v4:10.0.0.0/8,%v4:192.168.0.0/16,%v4:172.16.0.0/12,%v4:25.0.0.0/8,%v6:fd00::/8,%v6:fe80::/10
 oe=off
 protostack=netkey
 nhelpers=0
 interfaces=%defaultroute
 
conn vpnpsk
 auto=add
 left=$PRIVATE_IP
 leftid=$PUBLIC_IP
 leftsubnet=$PRIVATE_IP/32
 leftnexthop=%defaultroute
 leftprotoport=17/1701
 rightprotoport=17/%any
 right=%any
 rightsubnetwithin=0.0.0.0/0
 forceencaps=yes
 authby=secret
 pfs=no
 type=transport
 auth=esp
 ike=3des-sha1
 phase2alg=3des-sha1
 dpddelay=30
 dpdtimeout=120
 dpdaction=clear
 EOF
 
cat > /etc/ipsec.secrets <<EOF
 $PUBLIC_IP %any : PSK "$IPSEC_PSK"
 EOF
 
cat > /etc/xl2tpd/xl2tpd.conf <<EOF
 [global]
 port = 1701
 
;debug avp = yes
 ;debug network = yes
 ;debug state = yes
 ;debug tunnel = yes
 
[lns default]
 ip range = 192.168.42.10-192.168.42.250
 local ip = 192.168.42.1
 require chap = yes
 refuse pap = yes
 require authentication = yes
 name = l2tpd
 ;ppp debug = yes
 pppoptfile = /etc/ppp/options.xl2tpd
 length bit = yes
 EOF
 
cat > /etc/ppp/options.xl2tpd <<EOF
 ipcp-accept-local
 ipcp-accept-remote
 ms-dns 8.8.8.8
 ms-dns 8.8.4.4
 noccp
 auth
 crtscts
 idle 1800
 mtu 1280
 mru 1280
 lock
 connect-delay 5000
 EOF
 
cat > /etc/ppp/chap-secrets <<EOF
 # Secrets for authentication using CHAP
 # client server secret IP addresses
 
$VPN_USER l2tpd $VPN_PASSWORD *
 EOF
 
iptables -t nat -A POSTROUTING -s 192.168.42.0/24 -o eth0 -j MASQUERADE
 echo 1 > /proc/sys/net/ipv4/ip_forward
 
iptables-save > /etc/iptables.rules
 
cat > /etc/network/if-pre-up.d/iptablesload <<EOF
 #!/bin/sh
 iptables-restore < /etc/iptables.rules
 echo 1 > /proc/sys/net/ipv4/ip_forward
 exit 0
EOF
 
service ipsec start
service xl2tpd start
chkconfig ipsec on
chkconfig xl2tpd on

As promised, here are some details

• Lines 4-6 defines your security credentials for the VPN.  They must be changed before executing this script.
• Line 12 uses yum to install IPSec & L2TP implementation (OpenSWAN and xl2tpd) from the Amazon’s provided EPEL repository
• Lines 14-93 creates IPSec and L2TP configuration files, reusing the credentials you provided at the head of the script.
• Lines 95-96 setup proper network NATing
• Lines 98-105 ensure the network NATing settings will be restored in case the network interface is shutdown and up again.
• Finally, lines 107-110 start required services and ensure they will be restarted in case of reboot.

Congrats for those of you still reading.  You now should have a valid VPN server running in the cloud.  If everything went well, you should now be able to configure your VPN client.

How to connect from Mac OS X ?

Once the server is up and running, you simply add a VPN interface in your Network Preferences

Then, use the public DNS hostname as server address and your username, as shown below

Finally, click on “Authentication Settings” to enter the shared secret and your password.

Then click on Apply, then Connect

If everything is OK, you should connect to your new VPN Server.

How to be sure you’re connecting through the VPN ?

The easiest way to check that indeed all your network traffic is routed through the VPN tunnel is to connect to one of the many IP Address Geolocalisation web sites.

The web site I found on Google reported an Amazon IP address from Ireland, which is the geographical region I choose to deploy my VPN server.

A note for Windows users

Microsoft published an extensive technical note describing the details of setting up a IPSec client on Windows.

Also, Windows does not support IPsec NAT-T by default, which is used whenever the server is behind a NAT (as in this case). You have to add a registry key to enable this – see http://support.microsoft.com/kb/926179/en-us (still applies to Windows 8)

How to hook up a DNS alias to avoid to change client configuration ?

Every time you will startup a new VPN server, you will need to enter its public DNS name to your VPN client configuration.  It is possible to avoid this if you have a domain name of your own, just by creating a DNS CNAME record pointing to the public DNS address of your server, such as

vpn.mydomain.com CNAME ec2-176-34-71-204.eu-west-1.compute.amazonaws.com.

If you are using Amazon’s Route 53 DNS service, this step can be entirely automated using scripts.  More about this in another article.

Congrats if you manage to read this article to the end.  Once again, the script source code is available on GitHub.

Enjoy !

This year I will celebrate my 10th Devoxx attendance !  And for the first time I will have the pleasure to host two talks.

On Monday at 18:05 (Room #9), during a “Tools in Action” session, my colleague Abdoul and myself will build, live in front of the audience, a mobile application allowing to take pictures and capture geo localisation information and to send these for publishing on a web site. This demo will be built with open-source frameworks like DoJo and Apache’s Cordova, using IBM’s Worklight development IDE.  The architecture for this demo is depicted here under.

On Wednesday at 16:40 (Room #6), during the Conference, my colleague Eric and myself will demonstrate how IBM Rational Team Concert can be used to manage the lifecycle of a mobile application development, from capturing requirements to tests execution, changes and bugs management etc …

The rest of the time, you will find me on IBM’s booth in the exposition ground floor.

For those not knowing what Devoxx ambiance is, check out this nice video. Devoxx is sold out (again) this year : 3400 attendees from 40 different countries.

Liberty Profile is a standalone Java container.  It is not designed to be included in larger deployments based on WebSphere Application Server  ND cells.

However, Liberty Profile can take benefit of a shared persistence engine to store HTTP Session data. This allows two or more independent Liberty Profile instances to share a common user session for web applications.  When one instance fails, the surviving instances can continue to serve user requests as-is nothing happened.

Persistent data store might be a relational database (such as Derby used for development purposes) or a in-memory data grid. In-Memory Data Grid are software solutions providing in-memory data storage, replicated across different containers (or machines). Many IMDG solutions are available from different vendors or in open-source.  Most common ones are MemCached, Terracotta (Software AG), Coherence (Oracle) and IBM’s WebSphere eXtreme Scale.

If you are totally new to eXtreme Scale, I would recommend to read some basic information about its architecture before continuing to read this article.

Configuring WebSphere Application Server (WAS – full profile) to store HTTP Session in a eXtreme Scale container is a matter of three clicks in WAS admin console.  It is slightly more complicate with Liberty Profile, just a few configuration steps described below.

There are four different ways to install eXtreme Scale (XS) with Liberty :

• Run XS Container in a separate JVM or separate machine than Liberty Profile
• Run XS Container inside the same JVM as Liberty Profile
• Use Liberty Profile as client for an XS container
• Configure Liberty Profile to store HTTP Session data to an XS container

In this article, I will show you how to configure Liberty Profile to

1. Start an XS server within the same JVM as Liberty profile
2. Store HTTP Session data in this in-memory data grid,allowing to create clusters of Liberty Profile Instances

My final architecture is depicted in the image below.

0. Download and Install Liberty Profile and eXtreme Scale for Liberty Profile (both solutions are available at no charge from IBM – with forum based and peer-to-peer support only).

• Liberty Profile installation is described in my previous blog entry.
• eXtreme Scale for Liberty Profile installation is just a matter of unzipping the file in the directory above wlp

1. Create two servers instances

cd wlpBLOG
sst:wlpBLOG sst$ ./bin/server create ServerONE
Server ServerONE created.
sst:wlpBLOG sst$ ./bin/server create ServerTWO
Server ServerTWO created.

2. Change default HTTP Port in both server.xml so that the two instances can run in parallel

<httpEndpoint host="localhost" httpPort="9080" httpsPort="9443" id="defaultHttpEndpoint"/>

3. Add two features in server.xml for each server.  One to tell Liberty to run an XS server embedded.  And one to tell Liberty to use XS as HTTP Session store for web applications.

<!-- Enable features -->
<featureManager>
   <feature>jsp-2.2</feature>
   <feature>localConnector-1.0</feature>
   <feature>eXtremeScale.server-1.0</feature>
   <feature>eXtremeScale.web-1.0</feature>
</featureManager>

4. Configure the the WXS container inside Liberty Profile : add WXS configuration in Liberty Profile

<!-- Configuration for XS Server -->
<xsServer isCatalog="true" serverName="XS_ServerONE"/>
 
<!-- Configuration for Web Application XS HTTP Session data storage -->
<xsWebApp catalogHostPort="localhost:2809"
    objectGridType="REMOTE" 
    replicationInterval="0"
    reuseSessionId="true"
    securityEnabled="true"
    sessionTableSize="0"/>

5. Configure the the WXS container inside Liberty Profile : add XML configuration files in WLP runtime directory

In the directory WLP_HOME/usr/servers/ServerONE, create a “grids” directory and drop those two files

deployment.xml

<?xml version="1.0" encoding="UTF-8"?>
<deploymentPolicy xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://ibm.com/ws/objectgrid/deploymentPolicy ../deploymentPolicy.xsd"
xmlns="http://ibm.com/ws/objectgrid/deploymentPolicy">
 
<objectgridDeployment objectgridName="session">
<mapSet name="sessionMapSet" numberOfPartitions="47" minSyncReplicas="0" maxSyncReplicas="0" maxAsyncReplicas="1" developmentMode="false" placementStrategy="FIXED_PARTITIONS">
<map ref="objectgridSessionMetadata"/>
<map ref="objectgridSessionAttribute.*"/>
<map ref="objectgridSessionTTL.*"/>
</mapSet>
</objectgridDeployment>
</deploymentPolicy>

objectgrid.xml

<?xml version="1.0" encoding="UTF-8"?>
<objectGridConfig xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://ibm.com/ws/objectgrid/config ../objectGrid.xsd"
xmlns="http://ibm.com/ws/objectgrid/config">
<objectGrids>
<objectGrid name="session" txTimeout="30">
<bean id="ObjectGridEventListener" className="com.ibm.ws.xs.sessionmanager.SessionHandleManager"/>
<backingMap name="objectgridSessionMetadata" pluginCollectionRef="objectgridSessionMetadata" readOnly="false" lockStrategy="PESSIMISTIC" ttlEvictorType="LAST_ACCESS_TIME" timeToLive="3600" copyMode="COPY_TO_BYTES"/>
<backingMap name="objectgridSessionAttribute.*" template="true" readOnly="false" lockStrategy="PESSIMISTIC" ttlEvictorType="NONE" copyMode="COPY_TO_BYTES"/>
<backingMap name="objectgridSessionTTL.*" template="true" readOnly="false" lockStrategy="PESSIMISTIC" ttlEvictorType="LAST_ACCESS_TIME" timeToLive="3600" copyMode="COPY_TO_BYTES"/>
</objectGrid>
</objectGrids>
<backingMapPluginCollections>
<backingMapPluginCollection id="objectgridSessionMetadata">
<bean id="MapEventListener" className="com.ibm.ws.xs.sessionmanager.MetadataMapListener"/>
</backingMapPluginCollection>
</backingMapPluginCollections>
</objectGridConfig>

6. Tell Liberty’s session manager to reuse the same session ID for all user’s requests, even if handled by different JVM (See Liberty’s documentation for more details)

<httpSession idReuse="true"/>

7. Start Liberty Profile

sst:wlpBLOG sst$ ./bin/server start ServerONE
Server ServerONE started with process ID 11769.

In the logs, wait for the following line

[AUDIT ] CWWKF0011I: The server ServerONE is ready to run a smarter planet.

8. Create & Deploy a simple JSP file for testing

Create a Dynamic Web Project in Eclipse, and add the following index.jsp page

<%@page contentType="text/html" pageEncoding="UTF-8"%>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
 
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Liberty Profile Cluster Demo</title>
</head>
<body>
<h1>Liberty Profile - eXtreme Scale HTTP Session Demo!</h1>
<%

Integer count;
Object o = session.getAttribute("COUNT");
if (o != null) {
count = (Integer) o;
count = count + 1;
} else {
count = 1;
}
session.setAttribute("COUNT", count);

%>
<h3>This counter is increased each time the page is loaded.  Its value is stored in the <code>HttpSession</code></h3>
<h3><font color="#FF0000">Counter = <%=count%></font></h3>
<h4>Page server by cluster instance : <font color="#FF0000"><b><%= System.getProperty("wlp.server.name") %></b></font></h4>
<br/>
Page generated at = <%=new java.util.Date().toString()%><br/>
<br/>
</body>
</html>

Then deploy the WAR to the server instance (example of creating a WAR and deploying it to Liberty is given in my previous blog post)

9. Test, open your favorite browser and connect to http://localhost:9080/

You should see the following screen

Each time you will refresh the page (CTRL-R), the counter should be increased by one

Congrats, you have your first instance up and running, let’s now configure a second instance.

Repeat Steps 2-7 on a second Liberty instance to create a second cluster member.  Remember to change the following

• The name of the instance
• The HTTP and HTTPS ports used by Liberty Profile (step 2 above)
• The WXS configuration – only one catalog server is needed (step 3 above, change isCatalog=”no”)
• You do not need to copy the XML files in the grids directory of the second instance (step 5) – This is only required on the instance running XS’ Catalog Server

Then deploy your test application to instance #2.  To test your application, point your browser to

http://localhost:9081/<YOUR APPLICATION NAME>

You should see a page similar to the one shown at step 9 above.  Try to alternatively reload the page from ServerONE and the page from ServerTWO : you should see the session counter to increase in a sequence across the two server instances.

You’ve just created your first Liberty Profile cluster with two instances and a shared in-memory grid for HTTP session storage.

I leave you as an exercise to install and configure a load balancer in front of these two instances.  Hint : I am using the open-source balance for demo / test purpose.

If you find errors / typos in this (long) article, let me know – I will fix them – Thanks !

Enjoy !

Official JAX-RS / Liberty profile is available on IBM Documentation web site.  When developing or debugging REST based services, it is always good to know that IBM’s WebSphere Liberty profile is using Apache’s Wink implementation behind the scene.

Unlike some other Java based application servers (this one and this one for example), WebSphere Liberty Profile does not perform many under covers magical for you, in particular it does not register an application context, you will need to write (one line of) code to do that.

That being said, the process is quite similar for every application server and IDE :

1. Create a web based project

Choose a Project Name, select the runtime for deployment and uncheck the “Create EAR” option

2. add a POJO class that will serve as “resource”

Select a package name and class name.

Type the following code :

import javax.ws.rs.core.Context;
import javax.ws.rs.core.UriInfo;
import javax.ws.rs.PathParam;
import javax.ws.rs.Consumes;
import javax.ws.rs.PUT;
import javax.ws.rs.Path;
import javax.ws.rs.GET;
import javax.ws.rs.Produces;

@javax.ws.rs.ApplicationPath("resources")
@Path("/test")
public class Demo extends javax.ws.rs.core.Application {

    @Context
    private UriInfo context;

    @GET
    @Produces("application/xml")
    public String getXml() {
        return "<xml>Hello Rest World !</xml>";
    }

    @PUT
    @Consumes("application/xml")
    public void putXml(String content) {
    }
}

3. add your business code for the PUT and GET methods

4. Before deploying – Add JAX-RS “feature” to your server configuration

This will tell the Liberty kernel to load the JAX-RS server side implementation. You do not need to restart your server when adding / removing features.

5. Deploy and Test

At this stage, Eclipse’s browser will open on the application default URL and will display an error message.  This is normal as we did not define a landing page or default servlet in this project (index.jsp or index.html) for example.

To access the REST web service, use this URL pattern :

http://<hostname>:<port number>/<project name>/<application path>/<path>

which translates for this example to

http://localhost:9080/TestREST/resources/test

Et voilà, you just created, deployed and tested your first REST based web service on WebSphere Liberty Profile.

Enjoy !

On my i7 – quad core – machine, WAS Liberty starts in less than 1 sec.  With not application deployed.

Installing the runtime is as easy as unzipping a file on your drive, here are the steps

1. download from wasdev.net (46 Mb only)
2. unzip

   java -jar wlp-developers-8.5.0.0.jar

   After displaying and approving the distribution license, you will be ready for the next step

3. Optional : create a server instance (an instance “defaultServer” is created for you automatically, this step is optional)

   # cd wlp
   # chmod u+x bin/server
   # ./bin/server create MyInstance
   Server MyInstance created.

4. start it

   # ./bin/server start MyInstance

   Or just this line to start the default instance

   #./bin/server start

   Server MyInstance started with process ID 59946.

Now that you have the runtime, you are ready to install the tooling to manipulate it from Eclipse.

1.  Start Eclipse (Indigo or Juno)
2. Open Eclipse MarketPlace
   
3. Search for “liberty” and click on “Install”
   
4. In the “Eclipse” menu, click on “Preferences”
5. In the “Preferences” pane, select “Server”, then “Runtime Environment” and click on “Add”
   
6. Select “WebSphere Application 8.5 Liberty Profile”
   
7. Give the name you want, point to your Installation directory (see bullet 2 in the installation instructions above) and click “Finish”
   
8. Switch to the “Server” window in the “Java EE” perspective
9. Right-click – New -> Server, choose your newly created runtime instance
   
10. Don’t leave the “Server” window, right click on the server name and choose “Start”
    

The “Console” window should automatically open, and within a few seconds, you should see the following line to appear :

Launching default (wlp-1.0.0.20120428-1251/websphere-kernel_1.0.0) on Java HotSpot(TM) 64-Bit Server VM, version 1.7.0_07-b10 (en_US)
[AUDIT   ] CWWKE0001I: The server default has been launched.
[AUDIT   ] CWWKZ0058I: Monitoring dropins for applications. 
[AUDIT   ] CWWKF0011I: The server default is ready to run a smarter planet.

You have now a fully functional WebSphere Liberty profile installed and the corresponding tooling in Eclipse.  The tooling allows you to stop/start the application server, but also to manage its configuration and, obviously, to deploy applications on it.

In the next blog entry, I will show you how to deploy a REST based web service on Liberty

Enjoy !

Step #0 : be sure to have the lastest Android SDK installed.

Step #1 : Install Black Rose bootloader.  Although theoretically possible, I couldn’t manage to do this from Mac OS X.  Black Rose distribution provides a Linux and Windows binary to automate the process to the maximum.  Just launch it as root, look at your screen, wait for a couple of reboot … done !

I lost an hour because I did not start Black Rose as root on linux and it blocked during the process on

<waiting for the device>.

Be sure to start Black Rose as root !

I used a Virtual Machine with Ubuntu Natty and attached the Nexus One USB to the Virtual Machine – worked like a charm.

Step #2 : apply a small patch to HBoot, required for Jelly Bean

adb reboot bootloader
fastboot flash hboot hboot_jellybean_260-8-168.nb0
fastboot reboot-bootloader

You should see “Jelly Bean” on the second line

Step #3 : Flash Jelly Bean.  I downloaded the TAR file (not the ZIP)

First wipe out everything from the device

fastboot erase userdata
fastboot erase cache

Then flash the system

tar -xf <release>.tar.xz
fastboot erase system
fastboot flash system system.img
fastboot erase boot
fastboot flash boot boot.img

Step #4 : Install Google App.

At this stage, Jelly Bean should start on your Nexus.  It took me a while to realize that Google Apps are not installed by default.  Consequence : Contacts are not synced with your Google account and – most importantly – no Google Play ! So it is not possible to install additional applications. You’ll have to install them yourself to get access to Google Now, Google Accounts (and synchro), Maps and Google Play !

Apps are available on XDA Forums as well (GApps 7/13 at the time of this writing)

Step #4.1 : I installed GApps using a custom recovery application : ClockWork.  Download and installation instructions are here.

Step #4.2 : reboot in recovery mode to access ClockWork, then

use the menu system to put the Nexus One in USB Storage mode

• Copy GApps.zip to the flash card
• Use ClockWork’s menu to install GApps
• Reboot

and voilà you should have Jelly Beans + Google Apps running on your Nexus One.

A few things are not working so far :

• the Camera
• the trackball button to wake up the device

But the package is alpha software, so do expect updates and improvements in the coming weeks

Enjoy !

[UPDATE]
I upgraded with 20121113 nightly build (resuming my procedure from Step #3).  I used the simplified update with

 fastboot update Evervolv-perdo-3.1.0-passion-nightly-20121113-fastboot-update.zip

Also, clockwork do not need to be re-installed, it still lives in the recovery partition.

The system is much more responsive.  The camera and trackball button work as expected.

I also used M2SD to move app to SD card.  (be sure to partition the card as required)

It can store any type of data and provides REST API as long with Java (HashMap, JPA, Hibernate, Spring) APIs.  It also natively integrates with WebSphere Application Server and WebSphere Liberty Profile to cache HTTP session data.

It is supported on most platforms and – because it is a pure JavaSE application, it also works on Mac OS X, although this platform is not officially supported by IBM.

How to get started ?

• Download eXtreme Scale trial and unzip
• In a Terminal, go to product directory
• cd ObjectGrid/gettingstarted
• Run the Catalog Server
• ./runcat.sh
• Open another Terminal window and start an ObjectGrid server
• ./runcontainer.sh server0
• Repeat the last step to create several instances of ObjectGrid server
• Then experiment with client script.  It provides basic CRUD operations from command line
• ./runclient.sh i key value

Congrats, you managed to setup a multi instance grid, in-memory cache system on your Mac.

To further understand how it works and how you can programmatically interact with the cache, refer to eXtreme Scale documentation.

Next step will be to demonstrate how eXtreme Scale integrates with Liberty to create a multi instance cluster with shared HTTP Session. Stay Tuned.

Enjoy !

Therefore I collected information from documentation, blogs, internal IBM forums etc … to create the following list.

(click to enlarge )

This is *not* an official IBM document, just a compilation I gathered from various sources.  Please feel free to point me any missing or incorrect entries.

[UPDATE]

The official list of API supported in Liberty profile is now published in WAS 8.5 product documentation.

[/UPDATE]

Enjoy !

After Oracle abandoned the suite – and many other solutions in Sun Microsystems’ portfolio – the situation around Open Office is not easy to follow, let’s try to recap.

• A group of original developers from Sun, sponsored by Canonical, Novell, RedHat amongst others, forked OpenOffice and created LibreOffice.
• Oracle donated the original Open Office code base to the Apache Community, now published under an Apache v2 license
• Several large software editors have created derivative based on the OpenOffice code base, one of them being IBM’s Lotus Symphony (freely available)

Now that OpenOffice code base is not controlled by Oracle anymore, IBM decided to contribute its enhancement to the Apache OpenOffice project.  This is important news for all OpenOffice users.  This means that all improvements and changes made by IBM for Lotus Symphony will be made available for all in OpenOffice.

We are all looking forward the first release combining Apache OpenOffice and Lotus Symphony.

In the mean time, IBM released an iOS viewer application.  It allows you to view Open Document Format (ODF) text documents, presentations, and spreadsheets downloaded to your phone or tablet without the need for any network connection.

IBM OpenDocument Viewer for iOS is freely available on the Apple App Store.