In this blog post, I am going to share my experience working with Single Sign-On (a product from Red Hat) and will provide the detail information and instructions on how to configure different enterprise java applications with Single Sign-On using OpenID Connect (OIDC). This blog can be used by anyone who is interested in Single Sign-On implementation and be used by any developer to get started with Single Sign-On implementation quickly as possible.

This blog post will explain the high-level architecture (end-to-end request flow among applications), integration of SSO with JBoss EAP and BPM Suite, enabling SSO in Continuous Integration/Delivery and configuration of LDAP, AD and Kerberos for SSO.

Single Sign-On Implementation in Java

For securing application, the Red Hat Single Sign-On (RH-SSO) provides following options.

  • OpenID Connect (OIDC)
  • SAML
  • OAuth 2.0

Red Hat Single Sign-On comes with supports of SAML, OAuth 2.0 and OpenID Connect Connect out of the box.

OpenID Connect (OIDC) vs SAML

OIDC is a new protocol supported by Red Hat Single Sign-On comparing to old fully developed SAML protocol. Although SAML is proven and secure with supporting loads of use cases, however, in most cases, OIDC is still recommended. In this blog, we will use OIDC which is an extension to OAuth 2.0.

Architecture - End-to-End Application Request Flow

How does Single Sign-On Authentication work?

Single Sign-On provides the mechanism for authorization and authentication which enables users to authenticate once against one of our applications and doesn’t require to authenticate again for other applications.

Red Hat has released the Single Sign-On standalone product which provides the seamless integration and federated authentication for enterprise applications.

When applications are secured by Single Sign-On, a user is only required to authenticate the first time they access any of the secured resources. Upon successful authentication, the roles associated with the user retrieves from a local database (or external shared database, if using) and used for authorization of all other associated resources. This allows the user to access any further authorised resources without re-authenticating. After user logs out from an application, or an application invalidates the session programmatically, the process starts over.

The high level architecture request flow among different applications secured with Red Hat Single Sign-On is explained in figure below.

End-to-End Application Request Flow with Red Hat Single Sign-On

The end-to-end application request flow follows the similar pattern as any other SSO product. Any user tries to access an application first time without any valid authentication token will be redirected to Single Sign-On login page. Once user login successfully, the SSO token (containing token string along with user roles and other information) will need to set in a request header and passed to other applications.

Since the applications are already configured with SSO server, the SSO server first validates the session token from the incoming request and authenticates the user.

In case, the token expire or the user is not authorized to access the application, the Single Sign-On server will response with 401 unauthorized error code and user will be redirected to login page where user can obtain a valid token again.

A sample code to pass authentication token to other application is illustrated in example below.


private static void invokeServiceWithToken(HttpServletRequest req,
                             String uri) {
    CloseableHttpClient httpClient = null;
    try {
        httpClient = HttpClients.createDefault();            
        HttpGet get = new HttpGet(uri);
        KeycloakSecurityContext session = (KeycloakSecurityContext) req
            .getAttribute(KeycloakSecurityContext.class.getName());
            if (session != null) {
                get.addHeader("Authorization", "Bearer " +  
                       session.getTokenString());
            }

        HttpResponse response = httpClient.execute(get);
    } catch (Exception e) {
            throw new RuntimeException(e);
    } finally {
       if (httpClient != null)
           try {
               httpClient.close();
           } catch (Exception e) {
              throw new RuntimeException("Error closing 
                   HttpClient", e);
           }
    }
}

For more concrete examples, follow the link mentioned in section “Example Projects”.

The application flow explained in above Figure illustrate an example flow using different applications deployed in EAP, BPM Suite and Fuse, however, this can be generalised to any number of enterprise applications.

Single Sign-On Installation

The Red Hat Single Sign-On product can be downloaded from the RedHat. We need RedHat account with active subscription to download this license server. The Open Source version is available to download from here.

We will first download the Red Hat Single Sign-On server and will start the server with default configuration. However, we may need to change the default port for the server in case we are starting on the same host machine where other EAP servers are also running.

To change the default port to something else, add the appropriate JBoss binding offset in the standalone (or domain.xml) configuration.

    <system-properties>
         <property name="jboss.socket.binding.port-offset" value="1600"/>
    </system-properties>

After server start up successfully, visit the URL http://localhost:9680/ to ensure the Red Hat Single Sign-On server is running.

By default, the Red Hat Single Sign-On Server comes with the H2 database which is used for storing realm and user data. However, we can also configure an external database service. For instance, we need to configure the external database, if we are running in a cluster environment. Red Hat Single Sign-On supports loads of different databases like Oracle, MySql, PostgreSQL etc and the latest updates for the same list can be referred from Red Hat documentation.

The first thing, after installing server, would be to create an admin user account to manage the Red Hat Single Sign-On server.

To create admin user, use the URL http://localhost:9680/auth

After creating an admin user, we now log into the server and start playing with Red Hat Single Sign-On Server configuration. To login into Red Hat Single Sign-On Server, use the URL http://localhost:9680/auth/admin/

Realm

Realms are used to authenticate domain applications and are used to restrict access to the resources of different applications. Each realm is mapped to a domain and we can add as much as Realms for different domains. Red Hat Single Sign-On provides the default Master Realm and Admin user can use this to manage other realms. We can add multiple realms by using the Realms tab through the Administrator console. We can also configure different realms with different authentication protocol. For instance, Realm A can use OIDC protocol while Realm B can support SAML as an authentication mechanism. Similarly, each realm can be configured and associated to different LDAP or AD for user authentication.

Master Realm vs Application Realm

The master realm in Red Hat Single Sign-On is an exceptional realm and should handle differently compared to other application specific realms. The best practice is to create a separate new realm for securing application while keeping the Master Realm for management purpose only. Master Realm best be used as a place to create and manage other realms in our system.

Creating Realm and User

To create a new Realm, follow the link http://localhost:9680/auth/admin/ and use “Add Realm” option.

add realm from admin console

The detail steps found on the following link: creating a realm

Create new user by click “Users” menu and then “Add User”. This user is used for authentication and authorization for configured application which can be a web application, web service or simple mobile based application.

User Login and Logout

Since the user is managed against the realm, therefore, to login with the specific user, make sure the URL contains the correct realm.

For example, to login with user associated with “test” realm, the URL should be: http://localhost:9680/auth/realms/test/account.

Themes and Custom Login Page

Red Hat Single Sign-On Server comes with the default theme and login page. It is possible to create a custom login page specific to an application and configure/deploy in Red Hat Single Sign-On server. The server can also be themed for providing users with a seamless experience. Please refer to Red Hat documentation for more information.

Integrating Single Sign-On with JBoss EAP

Overview

This section explains the integration and configuration of Red Hat Single Sign-On with JBoss EAP. We assume that EAP is running in a standalone mode and therefore, we will modify standalone.xml file for the configuration. However, we need to either modify standalone-ha.xml or domain.xml depending on our operating mode.

Client Adapter

To secure application with Red Hat Single Sign-On, we need to install Red Hat Single Sign-On client adapters. The client adapter provides the tight integration of Red Hat Single Sign-On with any platform. It enables application to communicate and be secured by Red Hat Single Sign-On.

To begin, we will download the client adapter from RedHat website (account with active subscription is required). We need to choose the right version of adapter depending on the platform we are using. We install the client adapter using the simple CLI command as explained here

Register Application with Single Sign-On Server

Any application that needs to be secure with SSO, it has to be registered with Single Sign-On Server. We will first create and register the client using the Red Hat Single Sign-On admin console.

Login the admin console and use the “Clients” tab to register the client application. Please refer to the Red Hat documentation for detail steps on creating and register a new client application.

After registering the client, we configure the client installation using either the adapter subsystem configuration into the standalone.xml or Per WAR Keycloak configuration subsystem.

To configure using adapter subsystem configuration, open the standalone/configuration/standalone.xml file and add the following text:

    <subsystem xmlns="urn:jboss:domain:keycloak:1.1">
         <secure-deployment name="TestApp.war">
            <realm>testrealm</realm>
            <auth-server-url>localhost:9680/auth</auth-server-url>
            <public-client>true</public-client>
            <ssl-required>EXTERNAL</ssl-required>
            <resource>Test</resource>
         </secure-deployment>
    </subsystem>

Please note that we have to replace WAR name with the actual application WAR name. This will secure the application using the adapter subsystem as opposed to securing it through web.xml file inside each application WAR.

Please also note that we need to either change standalone-ha.xml or domain.xml depending on our operating mode.

To find detail information about how to secure application using “Per WAR Keycloak configuration”, please refer to the guideline here

Per WAR Keycloak vs Adapter Subsystem configuration

With Red Hat Single Sign-On, there are two ways to secure application.

  • Per WAR Keycloak configuration
  • Adapter Subsystem configuration

“Per WAR Keycloak configuration” requires adding config and editing files for each WAR separately. However, with the “Adapter Subsystem Configuration”, the changes need to be done only in standalone.xml file (or domain.xml).

To secure multiple application wars with Red Hat Single Sign-On, all the changes need to be performed in standalone.xml (or domain.xml) file. In the scenario where we want to automate the deployment process to different host environments, the “Adapter Subsystem configuration” provides better flexibility compared to “Per WAR Keycloak configuration” method as it is easy to configure environment variables in standalone.xml (or domain.xml) compared to each application WAR. Moreover, the standalone.xml (or domain.xml) is generally already configured to support different environments (System Test, UAT, Performance Test etc) and therefore, we may not need to perform any extra effort to accomplish this.

Single Sign-On and Continuous Integration

In this section, we will discuss the life cycle and explain how to enable the continuous integration or continuous delivery process with Single Sign-On.

Installation, Deployment, Configuration and Management

With Red Hat Single Sign-On, we can use any automation tool to enable CI/CD process. For example, I have used GoCD as Continuous Integration and continuous delivery process.

Figure below also explains the general continuous integration flow which can be implemented for any CI or CD tool like Jenkins, TeamCity or Hudson etc. In addition, we can also use deployment management tools like Ansible to automate the servers installation and infrastructure. However, this is not mandatory for CI process as installation or building infrastructure can be a onetime process.

Please note that the step 1, 2 and 3 in figure below either be done in Parallel or can follow any order.

SSO Continuous Integration and Continuous Delivery with Installation, Deployment, Configuration and Management Process

Step 1 is very basic in any CI process which builds the development projects and generates the required artifacts in order to deploy in Application Servers.

Step 2 simply installs the client adapter on EAP, BPM Suite and Fuse.

Step 3 explains the life cycle process including installation, administration and management of Red Hat Single Sign-On server.

We start with installing Red Hat Single Sign-On server. After installing server successfully, we perform some administration works. This includes configuring a Master Realm as well as creating an admin user. We also create application specific realm. Please note that we also use Master Realm for applications authentication, however, the good practice is that we should keep Master Realm for administration and management of other realms while application specific Realm for applications authentication.

Thereafter, we create and register client in Red Hat Single Sign-On server. To automate with CI process, I found “Adapter Subsystem Configuration” much better as this provides better flexibility to deploy on to different environment on the fly without changing any host URL or configuration manually. Therefore, after registering client, we also configure application using “Adapter Subsystem configuration” method to secure it with Red Hat Single Sign-On server.

Lastly, Step 4 deploys the apps in the respective application servers. Please note that if we are installing using “Per WAR Keycloak configuration” method, we may need to add keycloak.json in each application WAR artifact.

This is a general high-level request flow going across different application servers. This could easily be generalised for any kind and number of applications.

Configuring LDAP and Active Directory for Single Sign-On

Red Hat Single Sign-On supports LDAP and Active Directory users out of the box and supports integration with LDAP and Active Directory configuration for using external user databases. It works by importing the user on demand to its local database. We can always configure how to update the user information.

To configure with Red Hat Single Sign-On, click “User Federation” and then choose “Add Provider”. The below screen shots provide the detail options that is required to integrate with LDAP. Please use your LDAP configuration values and click “Sync all users” or “Sync changed users” to load/update the user information.

Single Sign On with LDAP and Active Directory integration configuration

More information about this be found here

By default, Red Hat Single Sign-On server import the LDAP users into the local database. To make sure that user information is up-to-date and sync with the LDAP server, we can use “Sync Setting” to configure synchronisation.

Integrating Single Sign-On with JBoss BPM Suite

We assume that JBoss BPM Suite is already installed and running. To configure JBoss BPM Suite with Single Sign-On, we will follow the steps below:

  • Install the client adapter on top of JBoss BPM Suite. The client adapter can be installed in a similar way as explained in section “Integrating Single Sign-On with JBoss EAP”.
  • Next, we will modify the standalone.xml as example below,

        <subsystem xmlns="urn:jboss:domain:keycloak:1.1">
             <secure-deployment name="business-central.war">
                <realm>testrealm</realm>
                <realm-public-key>ABC</realm-public-key>
                <auth-server-url>localhost:9680/auth</auth-server-url>
                <ssl-required>EXTERNAL</ssl-required>                      
                <resource>kie</resource>
                <credential name="secret">a7b4-35ea5d3dcc63</credential>
                <principal-attribute>preferred_username</principal-attribute>
                <enable-basic-auth>true</enable-basic-auth>
             </secure-deployment>
    
             <secure-deployment name="kie-server.war">
                <realm>testrealm</realm>
                <realm-public-key>ABC</realm-public-key>
                <auth-server-url>localhost:9680/auth</auth-server-url>
                <ssl-required>EXTERNAL</ssl-required>                      
                <resource>kie-execution-server</resource>
                <credential name="secret">bd57-fe1ae4781649</credential>
                <principal-attribute>preferred_username</principal-attribute>
                <enable-basic-auth>true</enable-basic-auth>
             </secure-deployment>
        </subsystem>
    

    Please note that we need to replace realm, realm-public-key and secret credential values with our actual realm values.

  • The realm client for Business Central and kie-server in Red Hat Single Sign-On as shown in the figure below: RH-SSO realm client for Business Central

    RH-SSO realm client for Kie Server/>

After performing the above changes, if we try to access Business Central, it should redirect to Red Hat Single Sign-On login screen. We can also secure the business central remote engine services API with Red Hat Single Sign-On. The detailed explanation can be referred here

Token Endpoints API

To generate the new token with Red Hat Single Sign-On server, we use any of the following:

  • Use the curl command to create a new token as below.
    curl GET -d "client_id=frontend" -d "username=mali" -d "password=mystrongpassword" -d "grant_type=password" http://localhost:9680/auth/realms/TestRealm/protocol/openid-connect/token
  • Use any Rest based tool to generate token. The screen shot below is from POSTMAN
  • POSTMAN Rest based tool for token generation

Please refer to API documentation regarding API details.

Example Projects

There are some example projects available on GitHub which helps to quickly demonstrate the Red Hat Single Sign-On integration with java applications. These projects are available here

In this blog post, I am going to explain the problem and the solution related to AJP connector executor leaking threads.

We may notice and detect that AJP thread pool with the thread counts will start increase continuously on application servers after upgrade patch with the latest Jboss EAP version. It looks like that the idle threads never recycle or reclaim even after keepalive-time. This behaviour will happen continuously even if the system will remain idle for a long period of time (without any user request). The application leak connections because it is not closing the AJP connection and returning them to the pool.

In the case of unbounded-queue-thread-pool executor configured and the default maximum-connections value being used, a new thread will create for every new request and will remain idle forever. As the number of AJP requests increase, the response from Jboss EAP may get slow eventually.

If we capture a java thread dump (kill -3 Java_ProcessID in Unix to generate thread dump) and check for the AJP threads, we can analyse that all AJP thread wait for requests and are hanging and holding on to the established keepalive connections.

"ajp-/XX.XX.XX.XX:9412-942" daemon prio=2 tid=xxxxx nid=XXX runnable [xxxxx] 
    java.lang.Thread.State: RUNNABLE 
 at java.net.SocketInputStream.socketRead0(Native Method) 
 at java.net.SocketInputStream.read(SocketInputStream.java:152) 
 at java.net.SocketInputStream.read(SocketInputStream.java:122) 
 at org.apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:1124) 
 at org.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:1206) 
 at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:438) 
 at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:420) 
 at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:926) 
 at java.lang.Thread.run(Thread.java:745)

The reason for this behaviour is that the AJP thread-pool connections by default have no timeout (default value of "DEFAULT_CONNECTION_TIMEOUT" is -1 for AJP, which means it never recycle) and therefore, the thread connection will persist permanently once established. The same behaviour detectable if we use httpd/mod_cluster in front of the JBoss which will invoke periodic pings to JBoss and fill its connection pool.

The solution is to add the following configuration in standalone.xml or domain.xml to limit the keepalive timeout value for AJP:

<system-properties>>
 <!-- default connection timeout is "-1" (means never timeout) --> 
 <property name="org.apache.coyote.ajp.DEFAULT_CONNECTION_TIMEOUT" 
         value="60000"/> 
</system-properties>

Google introduces new Gmail API which enables developers to use Gmail features into applications and make it easier for others to integrate and use as Plug-in.

According to Google:

The Gmail API gives you flexible, RESTful access to the user's inbox, with a natural interface to Threads, Messages, Labels, Drafts, and History.
The new API provides RESTful access to an inbox and supports CRUD operations on email threads, messages, labels etc.
The Gmail API can be used for a variety of applications including the cases where full inbox access is not required and the API can be much quicker compared to IMAP to search for a particular email.

The API provides fine-grained control to applications opposite to IMAP, where developers are required to obtain permission to the whole of an email inbox and therefore, will be restricted to:

  • Read messages from Inbox
  • Send messages
  • Amend the labels affect to messages
  • Search for specific messages

More Reading:

Java 8 introduces “Default Method” or (Defender methods) new feature, which allows a developer to add new methods to the Interfaces without breaking the existing implementation of these Interface. It provides flexibility to allow Interface define implementation which will use as default in the situation where a concrete Class fails to provide an implementation for that method.

Let consider small example to understand how it works:

public interface OldInterface {
    public void existingMethod();

    default public void newDefaultMethod() {
        System.out.println("New default method"
               + " is added in interface");
    }
}

The following Class will compile successfully in Java JDK 8:

public class OldInterfaceImpl implements OldInterface {
    public void existingMethod() {
     // existing implementation is here…
    }
}

If we create an instance of OldInterfaceImpl:

OldInterfaceImpl obj = new OldInterfaceImpl ();
// print “New default method add in interface”
obj.newDefaultMethod();  

Why Default Method?

Reengineering an existing JDK framework is always very complex. Modify one Interface in JDK framework breaks all Classes that extends the Interface which means that adding any new method could break millions of lines of code. Therefore, default methods have introduced as a mechanism to extending Interfaces in a backward compatible way.

Default methods can be provided to an Interface without affecting implementing Classes as it includes an implementation. If each added method within Interface defined with implementation then no implementing Class is affected. An implementing Class can override the default implementation provided by the Interface.

For Java 8, the JDK collections have been extended and forEach method is added to the entire collection (which work in conjunction with lambdas). With conventional way, the code looks like below:

public interface Iterable<T> {
    public void forEach(Consumer<? super T> consumer);
}

Since this result each implementing Class with compile errors, as a result, a default method added with a required implementation in order that the existing implementation should not be changed. The Iterable Interface with the Default method is below:

public interface Iterable<T> {
    public default void forEach(Consumer
                   <? super T> consumer) {
        for (T t : this) {
            consumer.accept(t);
        }
    }
}

The same mechanism has been used to add Stream in JDK Interface without breaking the implementing Classes.

When to use Default Method over Abstract Classes

Abstract Classes versus Interfaces in Java 8

After being introduced Default Method, the Interfaces and abstract Classes seems similar, however, they still different concept in Java 8. Abstract Class can define constructor. They are more structured and can have a state associated with them. While in contrast, default method can be implemented only in the terms of invoking other Interface methods, with no reference to a particular implementation's state. Hence, both use for different purposes and choosing between two really depends on the scenario context.

Default Method and Multiple Inheritance Ambiguity Problems

Since java Class can implement multiple Interfaces and each Interface can define default method with same method signature, therefore, the inherited methods can conflict with each other.

Consider below example:

public interface InterfaceA {  
       default void defaultMethod(){  
           System.out.println("Interface A default method");  
    }  
}

public interface InterfaceB {
   default void defaultMethod(){
       System.out.println("Interface B default method");
   }
}

public class Impl implements InterfaceA, InterfaceB  {
}

The above code will fail to compile with the following error, java: class Impl inherits unrelated defaults for defaultMethod() from types InterfaceA and InterfaceB

In order to fix this class, we need to provide default method implementation:

public class Impl implements InterfaceA, InterfaceB {
    public void defaultMethod(){
    }
}

Further, if we want to invoke default implementation provided by any of super Interface rather than our own implementation, we can do so as follows:

public class Impl implements InterfaceA, InterfaceB {
    public void defaultMethod(){
        // existing code here.
        InterfaceA.super.defaultMethod();
    }
}
We can choose any default implementation or both as part of our new method.

Difference between Default Method and Regular Method

Default Method is different from the regular method in the sense that default method comes with default modifier. Additionally, methods in Classes can use and modify method arguments as well as the fields of their Class but default method, on the other hand, can only access its arguments as Interfaces do not have any state.

In summary, Default methods enable to add new functionality to existing Interfaces without breaking older implementation of these Interfaces.
When we extend an interface that contains a default method, we can perform the following action:

  • Not override the default method and will inherit the default method.
  • Override the default method similar to other methods we have overridden subclasses.
  • Redeclare default method as abstract, which will force subclasses to override it.

References

In this post, we are going to discuss the deployment problem that prevents JBoss Server to start properly.

Error starting Java JBoss server (in domain mode)

The JBoss Server logs, in this case, looks like below:

10:16:57,337 INFO [org.jboss.modules] JBoss Modules version 1.1.3.GA-redhat-1
10:16:57,498 INFO [org.jboss.msc] JBoss MSC version 1.0.2.GA-redhat-2

10:16:57,686 INFO [org.jboss.as] JBAS015899: JBoss EAP 6.0.1.GA
(AS 7.1.3.Final-redhat-4) starting

10:16:58,310 INFO [org.jboss.as.server.deployment.scanner] JBAS015014: Re-attempting failed deployment abc.warploy

10:16:58,340 INFO [org.jboss.as.server.deployment.scanner] JBAS015003: Found abc.war in deployment directory. To trigger deployment create a file called abc.war.dodeploy

10:16:58,348 ERROR [org.jboss.as.controller.management-operation] JBAS014613: Operation ("add") failed - address: ([("deployment" => "abc-ABC.0.0.21.war")]) - failure description: "JBAS018717: No deployment content with hash 28c2ce34057a6bd5ebf2c28f9d114814faa66b8a is available in the deployment content repository for deployment 'abc-ABC.0.0.21.war'.

This is a fatal boot error. To correct the problem, either restart with the --admin-only switch set and use the CLI to install the missing content or remove it from the configuration, or remove the deployment from the xml configuraiton file and restart."

10:16:58,351 ERROR [org.jboss.as.server.deployment.scanner] JBAS014654: Composite operation was rolled back

10:16:58,351 FATAL [org.jboss.as.server] JBAS015957: Server boot has failed in an unrecoverable manner; exiting. See previous messages for details.

10:16:58,358 INFO [org.jboss.as] JBAS015950: JBoss EAP 6.0.1.GA (AS 7.1.3.Final-redhat-4) stopped in 3ms

This error also occurs when we delete directory standalone\data.. and then try to redeploy the same application war. The JBoss server unable to delete entries defined in standalone.xml with the same hash for the deployed application.

The standalone.xml looks like below:

<deployments>
    <deployment name="abc-ABC.0.0.21.war" runtime-name="abc-ABC.0.0.21.war">
         <content sha1="28c2ce34057a6bd5ebf2c28f9d114814faa66b8a"/>
    </deployment>
</deployments>

Removing the entries manually for the application having defined hash solved this problem and JBoss server will start successfully without error.

Eclipse has recently announced the release of Eclipse Juno which brings new features such as Detecting resource leaks for both Java7 and Java6 etc.

Previous Plugins to Install

I firstly install all my plugins from previous eclipse version. The process is quite simple
  • import and install from existing installation
  • refer to old eclipse location
  • Eclipse will show the list of plugins available and then asked for installation
  • After accepting license, installation should complete smoothly.

New Features available

  • Java 7 coding support in an IDE. Since Java 7 contains many new feature including dynamically-typed languages support and other small enhancements from Coin project, eclipse Java development tooling will include support for these features of Java 7.
  • Detecting resource leaks of Closeable/Autocloseable resources. This feature is really cool and it also works with “old code”. The common IO and JDBC resources now implement the relevant interfaces and the warnings are good enough to be shown on these.
    Resource leaks can be occurred in the following situations:
         1. A resource opened but not closed
         2. A resource may not closed on all control flows
         3. A resource may not closed at a method exit point
         4. In a Java 7 program a resource is closed but the code could still be improved by using a try-with-resources statement.

  • The new version include the global search bar that gives user quick access to almost any Eclipse feature.
    Glabal search bar
  • Code Recommenders This is amazing feature that analyzes code of existing applications and extracts common patterns of how other developers have used. It helps deal with the complexity of large APIs using intelligent code completion attribute.

    More examples can be found from below links,

    http://eclipsesource.com/blogs/2012/06/26/code-recommenders-top-eclipse-juno-feature-2/
    https://code-recommenders.blogspot.com/

  • The addition of integrated debugging of JVM-based domain specific languages in the Xtext development framework, The below link provide extensive examples regarding the mentioned feature, http://eclipsesource.com/blogs/2012/06/25/xtext-xtend-top-eclipse-juno-feature-3/

  • Eclipse for Mobile Developers - now developers can use eclipse more easily with a variety of mobile SDKs, including the Google Android SDK

Amazon Elastic MapReduce (EMR) is a web service using which developers can easily and efficiently process enormous amounts of data. It uses a hosted Hadoop framework running on the web-scale infrastructure of Amazon EC2 and Amazon S3.

Amazon EMR removes most of the cumbersome details of Hadoop, while take care for provisioning of Hadoop, running the job flow, terminating the job flow, moving the data between Amazon EC2 and Amazon S3, and optimising Hadoop.

In this tutorial, we will first going to develop WordCount java example using a MapReduce framework Hadoop and thereafter, we execute our program on Amazon Elastic MapReduce.

Prerequisites

You must have valid AWS account credentials. You should also have a general familiarity with using the Eclipse IDE before you begin. The reader can also use any other IDE of their choice.

Step 1 – Develop Hadoop MapReduce WordCount Java Program

In this section, we will first develop an WordCount application. A WordCount program will determine how many times different words appear in a set of files.

  • 1. In Eclipse (or whatever the IDE you are using), create a simple Java project name "WordCount".
  • 2. Create a java class name Map and override the map method as follow:
    public class Map extends Mapper<longwritable, 
                               intwritable="" text,=""> {
     private final static IntWritable one = 
                              new IntWritable(1);
     private Text word = new Text();
    
     @Override
     public void map(LongWritable key, Text value, 
                         Context context)
         throws IOException, InterruptedException {
       String line = value.toString();
       StringTokenizer tokenizer = new 
                            StringTokenizer(line);
       while (tokenizer.hasMoreTokens()) {
           word.set(tokenizer.nextToken());
           context.write(word, one);
       }
      }
    }
    
  • 3.Create a java class name Reduce and override the reduce method as below:
    public class Reduce extends Reducer<text, 
                  intwritable,="" intwritable="" text,=""> {
     @Override
     protected void reduce(
       Text key,
       java.lang.Iterable<intwritable> values,
       org.apache.hadoop.mapreduce.Reducer<text, 
               intwritable,="" intwritable="" text,="">.Context context)
       throws IOException, InterruptedException {
      int sum = 0;
      for (IntWritable value : values) {
       sum += value.get();
      }
      context.write(key, new IntWritable(sum));
     }
    }
    
  • 4. Create a java class name WordCount and defined the main method as below:
    public static void main(String[] args) 
                          throws Exception {
      Configuration conf = new Configuration();
    
      Job job = new Job(conf, "wordcount");
      job.setJarByClass(WordCount.class);
    
      job.setOutputKeyClass(Text.class);
      job.setOutputValueClass(IntWritable.class);
    
      job.setMapperClass(Map.class);
      job.setReducerClass(Reduce.class);
    
      job.setInputFormatClass(TextInputFormat.class);
      job.setOutputFormatClass(TextOutputFormat.class);
    
      FileInputFormat.addInputPath(job, new Path(args[0]));
      FileOutputFormat.setOutputPath(job, new Path(args[1]));
    
      job.waitForCompletion(true);
    }
    
  • 5. Export the WordCount program in a jar using eclipse and save it to some location on disk. Make sure that you have provided the Main Class (WordCount.jar) during extracting the jar file as shown below:

jar
jar ready
                                 ur jar is ready!!

Step 2 – Upload the WordCount JAR and Input Files to Amazon S3

Now we are going to upload the WordCount jar to Amazon S3. First, visit the following URL: https://console.aws.amazon.com/s3/home Next, click “Create Bucket”, give your bucket a name, and click the “Create” button. Select your new S3 bucket in the left-hand pane. Upload the WordCount JAR and sample input file for counting the words.

Step 3 – Running an Amazon Elastic MapReduce Hadoop job

Running Hadoop WordCount example

Now that the JAR is uploaded into S3, all we need to do is to create a new Job flow. let's execute the below steps. (I encourage reader to check out the following link for details regarding each step, How to Create a Job Flow Using a Custom JAR )

  • 1. Sign in to the AWS Management Console and open the Amazon Elastic MapReduce console at https://console.aws.amazon.com/elasticmapreduce/
  • 2. Click Create New Job Flow.
  • 3. In the DEFINE JOB FLOW page, enter the following details:

    a. Job Flow Name = WordCountJob
    b. Select Run your own application
    c. Select Custom JAR in the drop-down list
    d. Click Continue

  • 4. In the SPECIFY PARAMETERS page, enter values in the boxes using the following table as a guide, and then click Continue.
    JAR Location = bucketName/jarFileLocation
    JAR Arguments =
    s3n://bucketName/inputFileLocation
    s3n://bucketName/outputpath

    Please note that the output path must be unique each time we execute the job. The Hadoop always create a folder with the same name specify here.

After executing a job, just wait and monitor your job that runs through the Hadoop flow. You can also look for errors by using the Debug button. The job should be complete within 10 to 15 minutes (can also depend on the size of the input). After completing a job, You can view results in the S3 Browser panel. You can also download the files from S3 and can analyse the outcome of the job.

Amazon Elastic MapReduce Resources