Artifactory with Multi-Module Gradle Project

Prerequisites

1. Oracle JDK8

Environment Details

1. OS – Ubuntu 16.04 x64 Server
2. Artifactory – oss-5.3.0

Problem

Publish your jar from Multi-module gradle project and use in other projects.

Steps

Install Artifactory

Start Artifactory

  • Goto artifactory home. (to extracted location)
  • run – bin/artifactory.sh

Setup repository

Your framework project parent/root build.gradle


buildscript {
  repositories {
    jcenter {
      url 'http://localhost:8081/artifactory/jcenter/'
    }
  }

  dependencies {
    //Check for the latest version here: http://plugins.gradle.org/plugin/com.jfrog.artifactory
    classpath "org.jfrog.buildinfo:build-info-extractor-gradle:4+"
  }
}

allprojects {
  apply plugin: 'maven'

  group = 'com.example'
  version = '1.0.0-SNAPSHOT'

  ext {
    springBootVersion = '1.5.2.RELEASE'
  }
}

subprojects {
  apply plugin: 'java'
  apply plugin: 'maven-publish'
  apply plugin: "com.jfrog.artifactory"

  sourceCompatibility = 1.8
  targetCompatibility = 1.8
  tasks.withType(JavaCompile) {
    options.encoding = 'UTF-8'
  }

  repositories {
    jcenter {
      url 'http://localhost:8081/artifactory/jcenter/'
    }
  }

  dependencies {
    testCompile group: 'junit', name: 'junit', version:'4.12'
  }

  artifactory {
    contextUrl = "${artifactory_contextUrl}" //The base Artifactory URL if not overridden by the publisher/resolver
    publish {
      repository {
        repoKey = 'example-framework-snapshot-local'
        username = "${artifactory_user}"
        password = "${artifactory_password}"
        maven = true
      }
      defaults {
        publications ('mavenJava')
      }
    }

    resolve {
      repository {
        repoKey = 'example-framework-snapshot'
        username = "${artifactory_user}"
        password = "${artifactory_password}"
        maven = true
      }
    }
  }

  publishing {
    publications {
      mavenJava(MavenPublication) {
        from components.java
      }
    }
  }
}

build.gradle for user of your Framework Code

Assuming your virtual repository is ‘example-framework-snapshot’

buildscript {
  ext {
    springBootVersion = '1.5.2.RELEASE'
  }
  repositories {
    maven {
      // Your virtual repository
      url 'http://localhost:8081/artifactory/example-framework-snapshot'
    }
  }
  dependencies {
    classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
  }
}

apply plugin: 'java'
apply plugin: 'maven'
apply plugin: 'war'
apply plugin: 'org.springframework.boot'

group = 'com.security.sample'
version = '0.0.1-SNAPSHOT'

description = """sample-security Webapp"""

sourceCompatibility = 1.8
targetCompatibility = 1.8

repositories {
  maven {
    url 'http://localhost:8081/artifactory/example-framework-snapshot'
  }
}

dependencies {
  compile ('com.example:example-security-extras:0.0.1-SNAPSHOT')
  compile ('org.springframework.boot:spring-boot-starter-web')
  compile ('redis.clients:jedis')
  testCompile ('junit:junit')
}

Advertisements

Sorting and Cross Join Problem – Spring Data JPA

Affected Version

Spring boot – 1.2.2
Spring data JPA version – 1.7.2
Spring data common – 1.9.2

Problem Statement

When you use custom query using @Query in your spring data JPA repository and pass sort parameter, where sort parameter is a child attribute; the query generated uses cross join instead of left outer join. This gives unexpected result.

Problem Sample


@Query("SELECT e FROM EventApprovalRequest e WHERE e.approvalRequestType = (:approvalRequestType) "
+ "AND e.status = (:status)")
Page getEventsForApproval(
@Param("approvalRequestType") int approvalRequestType, @Param("status") int status, Pageable pageable);

Solution

Pre-define left outer join in your query.

@Query("SELECT e FROM EventApprovalRequest e LEFT OUTER JOIN e.event LEFT OUTER JOIN e.event.director WHERE e.approvalRequestType = (:approvalRequestType) "
+ "AND e.status = (:status)")
Page getEventsForApproval(
@Param("approvalRequestType") int approvalRequestType, @Param("status") int status, Pageable pageable);

Get Enum type with id

Here is a simple technique to retrieve Enum type using custom ‘id’ property instead of name.

Define Contract

/**
 * Contract that will allow Types with id to have generic implementation.
 */
public interface IdentifierType<T> {
  T getId();
}

Apply Contract

public enum EntityType implements IdentifierType<Integer> {
  ENTITY1(1, "ONE), ENTITY2(2, "TWO");

  private Integer id;
  private String name;

  private EntityType(int id, String name) {
    this.id = id;
    this.name = name;
  }

  public static EntityType valueOf(Integer id) {
    return EnumHelper.INSTANCE.valueOf(id, EntityType.values());
  }

  @Override
  public Integer getId() {
    return id;
  }
}

Make use of contract (helper)

public enum EnumHelper {
  INSTANCE;

  /**
   * This will return {@link Enum} constant out of provided {@link Enum} values with the specified id.
   * @param id the id of the constant to return.
   * @param values the {@link Enum} constants of specified type.
   * @return the {@link Enum} constant.
   */
  public <T extends IdentifierType<S>, S> T valueOf(S id, T[] values) {
    if (!values[0].getClass().isEnum()) {
        throw new IllegalArgumentException("Values provided to scan is not an Enum");
    }

    T type = null;

    for (int i = 0; i < values.length && type == null; i++) {
        if (values[i].getId().equals(id)) {
            type = values[i];
        }
    }

    return type;
  }
}

Search Framework Idea – Java

Introduction

Writing code to implement search in application is a repetitive task, and need understanding of query language provided by underlying search engine. Search framework reduces the development effort and hides implementation complexity by leveraging good Object Oriented principles. It helps to make your design extensible and maintainable.

The solution built using following design patterns

  • Builder
  • Factory
  • Template
  • Strategy

 

Architecture

SearchFramework

Core Components

It consist of interfaces defining contract for searcher, and beans defining search request object format.

Code Sample

public interface Searcher {
   T search(SearchCriteria criteria);
}

public class SearchCriteria {
   private int groupCondition;
   private List searchFilters;
}

public class SearchFilter {
   private String fieldName;
   private String fieldValue;
   private int filterCondition;
}

Provider Integration

In this layer we do provider specific implementation, such that it is implemented once and reused.

Sample for JPA Searcher

public abstract class JPASearcher implements Searcher {
   @Autowired
   private PredicateBuilder predicateBuilder;

   protected Specification getSpecification(final SearchCriteria searchCriteria) {
     return new Specification() {

      @Override
       public Predicate toPredicate(Root root, CriteriaQuery criteriaQuery,
 CriteriaBuilder criteriaBuilder) {
         criteriaQuery.distinct(true);
         return predicateBuilder.build(searchCriteria, criteriaBuilder, root);
       }

   };
}

JPA Predicate builder

@Component
public class PredicateBuilder {
   @Autowired
   private PredicateConditionBuilderFactory conditionBuilderFactory;
   
   public Predicate build(SearchCriteria criteria, CriteriaBuilder cb, Root root) {
     …
     for (SearchFilter searchFilter : criteria.getSearchFilters()) {
       …
     }
     if (criteria.getGroupCondition() == GroupCondition.AND) {
       predicate = cb.and(predicates.toArray(new Predicate[predicates.size()]));
     } else {
       predicate = cb.or(predicates.toArray(new Predicate[predicates.size()]));
     }
   return predicate;
   }
}
 

JPA Condition Builder

public interface PredicateConditionBuilder {

 Predicate build(SearchFilter filter, CriteriaBuilder criteriaBuilder, Root root,
 final DistinctJoinHolder joinHolder);

}

Sample Implementation

public class IsNotNullBuilder implements PredicateConditionBuilder {

private final ConditionBuilderHelper builderHelper = ConditionBuilderHelper.INSTANCE;

@Override
public  Predicate build(SearchFilter filter, CriteriaBuilder criteriaBuilder, Root root,
 final DistinctJoinHolder joinHolder) {
    String fieldName = filter.getField().getSearchFieldName();
    GroupCondition groupCondition = filter.getSearchCriteria().getGroupCondition();
    Path path = builderHelper.getPath(fieldName, root, groupCondition, joinHolder);
    return criteriaBuilder.isNotNull(path);
  }
}

JPA Condition Builder Factory

@Component
public class PredicateConditionBuilderFactory {

  private Map builders;

  @PostConstruct
  protected void init() {
     builders = new HashMap();
     builders.put(FilterCondition.EQUAL, new EqualBuilder());
     builders.put(FilterCondition.NOT_EQUAL, new NotEqualBuilder());
     builders.put(FilterCondition.GT, new GreaterThanBuilder());
     builders.put(FilterCondition.LT, new LessThanBuilder());
     builders.put(FilterCondition.EQ_GT, new GreaterThanEqualBuilder());
     builders.put(FilterCondition.EQ_LT, new LessThanEqualBuilder());
     builders.put(FilterCondition.BETWEEN, new BetweenBuilder());
     builders.put(FilterCondition.LIKE, new LikeBuilder());
     builders.put(FilterCondition.IN, new InBuilder());
     builders.put(FilterCondition.IS_NULL, new IsNullBuilder());
     builders.put(FilterCondition.IS_NOT_NULL, new IsNotNullBuilder());
   }

  public PredicateConditionBuilder getBuilder(FilterCondition filterCondition) {
     return builders.get(filterCondition);
  }

}

Search Configuration

It is not used directly by core framework, but needed by various search utilities – for example search criteria builder utility may want to get list of fields supported in application’s basic search, or may be for validation configuration.

Search Utilities

This layer consist of various search supporting utilities –

  • Search criteria builder & validator.
  • Search filter builder & validator.

Simple JPA Searcher implementation (Search Services)

@Component
public class EventSearcher extends JPASearcher {
   @Autowired
   private EventRepository repository;

   @Override
   public List search(SearchCriteria criteria) {
     return repository.findAll(getSpecification(criteria));
   }
}

All you have to do is

  • Extend JPASearcher and provide entity to search.
  • Provide jpa repository for entity.
  • Call search with search criteria.

Simple Lambda – Java8

public class LambdaSample {
  protected interface SimpleCalc {
    int operation(int a, int b);
  }
  public static void main(String[] args) {
    /*
     * Old style
     */
    SimpleCalc add = new SimpleCalc() {
    @Override
    public int operation(int a, int b) {
      return a + b;
    }
  };
  System.out.println(add.operation(2, 5));
  /*
   * New style - Lambda Expression
   */
  SimpleCalc multiply = (a, b) -> a * b;
  System.out.println(multiply.operation(2, 5));
  }
}

In the above code you can see the old vs new style of anonymous function creation. Lambda expression makes it short, concise and readable. [its look readable to me :)]

Building Cloudera cdh5.1.3 – Hadoop Native – [Ubuntu 14.04 Server x64]

WARN: Unable to load native-hadoop library for your platform… using builtin-java classes where app

On starting Cloudera cdh5 hadoop if you get above warning – then either your hadoop native is not on path or you don’t have native library available in your distribution.

For case 1: solution is simple.

export HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=$HADOOP_HOME/lib/native"

For case 2: follow below steps

  1. Install jdk7 and maven [skip if already installed]
  2. sudo apt-get install build-essential
  3. apt-get install cmake
  4. apt-get install pkg-config
  5. apt-get install libcurl4-openssl-dev
  6. install protobuf-2.5.0.tar.gz
  7. Goto “hadoop-2.3.0-cdh5.1.3/src” folder and run
    mvn package -Pdist,native -DskipTests -Dtar

check “hadoop-2.3.0-cdh5.1.3/src/hadoop-dist/target/hadoop-2.3.0-cdh5.1.3/lib/native”
or you can use – “hadoop-2.3.0-cdh5.1.3/src/hadoop-dist/target/hadoop-2.3.0-cdh5.1.3.tar.gz”

Philosophy behind NodeJS

In this section I will try to present philosophical aspects behind NodeJS covering design principles and programming techniques inspired its development.

  1. Hollywood Principle: “Don’t call us we will call you”. To understand this here’s a small story – a struggling actor goes to some director and ask for role, currently the director don’t have any suitable assignment for him, and he don’t want to be bothered again and again, so he asked actor to leave his contact detail to his assistant and says – “don’t call me I will call you if some assignment is there”. This principle initiated the callback mechanism, whereby one task depending on other task for some processing, simply submit request along with a callback method/function to be called by other task when result is available. This way tasks do not block important resources like ‘thread’ in waiting.
  2. Asynchronous Call: To understand this first understand that ‘asynchronous’ is the state of being ‘not synchronised’; synchronous process generally wait for final result to be calculated whether success or failure. In asynchronous call you submit your request along with callback handler that must be triggered when result is prepared and returned, your process doesn’t wait rather continue with its flow. To achieve this ofcourse threads are required but that will be handled by nodejs for developer its single threaded programming model.
    Example: executeQuery("select empid from employee", resultHandler(data))
    Here your resultHandler is a callback handler.

    Please see http://blog.mixu.net/2011/02/01/understanding-the-node-js-event-loop/ for more detail on single thread model.

  3. Non-Blocking IO: NodeJS is suitable for IO intensive application rather computation intensive application (more on this can be found in google). Non-Blocking IO means your program execution flow is not blocked for any IO operation and this is also achieved through callback. Javascript support’s callback implementation but didn’t have any IO API, for node non-blocking IO implementation was added, which is available for developer out of the box.
    Example: executeQuery("select empid from employee", resultHandler(data));
    displayProcessingWindow();
    Here you can see displaying process running info to user is called after db call is made; and this will display process window because your execution flow is not blocked by the call of ‘executeQuery’, also you are not doing anything special to achieve this it will be handled out of the box.
  4. Multiple Request Single Thread: Javascript follows single thread programming model, means there is no way you can spawn new thread in your code/implementation, and this is done to achieve simple programming model. Parallel execution is handled out of the box. Node assume that any call other than IO will be completed immediately, that is why it’s not recommended for computation intensive tasks; as IO operations are non-blocking it will not halt your execution flow. (IO operations are handled parallel in the background using thread)

Conclusion: Hollywood principle is the key design principle behind non-blocking implementation, and callback and event loop is the technique to achieve asynchronous/non-blocking behaviour.

Purpose

  1. Node is designed to be used for applications that are heavy on input/output, but light on computation (computation intensive task should be delegated out of node).
  2. To leverage single thread programming model (for developers) to achieve simplicity, while handling complex multithreading in the background out of the box.
  3. To reduce memory overhead that is inherent in request per thread model.

References