• Spark Kafka Streaming Java program Word Count using Kafka 0.10 API

    Donnerstag, 12. Januar 2017 - 0:39
    Kafka API went through a lot of changes starting Kafka 0.9. Spark Kafka Streaming API also was changed to better support Kafka 0.9. i wanted to try that out so i built this simple Word Count application using Kafka 0.10 API. This blog entry does the same thing but using Scala.
    Blogeintrag
  • How to use ElasticSearch as storage from Hive in cloudera

    Sonntag, 1. Januar 2017 - 5:07
    .In the Using ElasticSearch as external data store with apache hive entry i talked about how you can create a table in Hive so that actual data is stored in ElasticSearch. Problem with that approach was that i had to pass the full path to elasticsearch-hadoop-hive-<eshadoopversion>.jar as parameter every time. hive -hiveconf hive.aux.jars.path=/opt/elastic/elasticsearch-hadoop-2.4.3/dist/elasticsearch-hadoop-hive-2.4.3.jar;
    Blogeintrag
  • How to implement cache (LRU Cache) using LinkedHashMap in java

    Donnerstag, 12. Januar 2017 - 6:08
    Recently i wanted to implement a simple Least recently used (LRU) cache in one my applications. But my use case is simple enough that instead of going for something ehcache i decided to build it on own by using java.util.LinkedHashMap As you can see from the code its very simple.
    Blogeintrag
  • Spark program to read data from RDBMS

    Donnerstag, 12. Januar 2017 - 6:48
    I wanted to figure out how to connect to RDBMS from spark and extract data, so i followed these steps. You can download this project form github First i did create Address table in my local mysql like this CREATE TABLE `address` ( `addressid` int(11) NOT NULL AUTO_INCREMENT, `contactid` int(11) DEFAULT NULL, `line1` varchar(300) NOT NULL, `city` varchar(50) NOT NULL, `state` varchar(50) NOT NULL, `zip` varchar(50) NOT NULL,
    Blogeintrag
  • How to reset consumer group offset

    Freitag, 3. November 2017 - 22:48
    First run describe on topic to check what it the current LAG its zero in this case bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group user.kafkaconsumer Now run following command to just preview what will be the next offset if you reset bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group user.kafkaconsumer --reset-offsets --to-earliest --all-topics
    Blogeintrag
  • Kafka how to reset number of partitions in a topics

    Freitag, 3. November 2017 - 23:19
    I wanted to figure out how to reset number of partitions in my topic in Kafka and I followed these steps I did create a sample topic called my-topic with single partition spatil$ bin/kafka-topics.sh --create --zookeeper localhost:2181 --topic my-topic --replication-factor 1 --partitions 1Created topic "my-topic". I used describe command to verify that my topic has actually single partition spatil$ bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic my-topic
    Blogeintrag
  • Liberty virtual Proof of Technology on Jan 11, 2017

    Mittwoch, 7. Dezember 2016 - 18:33
    WebSphere development team is winding up work on Liberty version 16.0.0.4. As usual we will be doing a webcast to explain all the great new stuff, and let you get your hands on a product download and a set of... Read More ›
    Blogeintrag
  • Performance testing IBM MQ in a Docker environment

    Dienstag, 3. Januar 2017 - 23:33
    At the end of 2016 I spent some time performance testing IBM MQ, Pivotal RabbitMQ and Apache ActiveMQ in a Docker environment. This project goal was similar to my earlier tests on the native OS described in this blog post,... Read More ›
    Blogeintrag
  • Competitive Study of IBM Bluemix vs. AWS for Moving WAS Workloads to the Cloud

    Samstag, 14. Januar 2017 - 4:45
    This blog discusses the results of a competitive study to measure the learning time, deployment time and cost to run a Web Application that is moved from an on-premises environment to a public cloud environment. The public cloud environments compared are WebSphere on Bluemix service and Amazon AWS using MidVision WAS AMIs.
    Blogeintrag
  • Hello World!

    Donnerstag, 2. Juli 2009 - 11:52
    Herzlich Willkommen im Integration Blog! Business Integration ist öde und trocken? Mit uns nicht! Wir lieben was wir tun und wollen das nicht länger für uns behalten. Heute werfen wir ein paar angestaubte Vorstellungen über Bord und schaffen Platz für Neues: Hier teilen wir mit Ihnen ab sofort Interessantes aus unserer täglichen Arbeit und der Branche - was bewegt sich und was bewegt uns. Sie möchten wir dabei haben: Ihre Kommentare sind für einen Dialog hilfreich und sehr erwünscht. Wir freuen uns auf einen gemeinsamen Informations- und Meinungsaustausch.
    Blogeintrag