The WebSphere Notes

31. März 2011 - 15:01

The WebSphere Notes (www.webspherenotes.com)  is a blog that has my study notes about WebSphere Application server administration and WebSphere Portal Server developer and administration certification.

 

Sunil has been in the IT industry for 10 years, worked with IBM Software Labs and was part of WebSphere Portal Server Development team for 4 years, and is now working for Ascendant Technology. Sunil has been working with WebSphere Portal since 2003. He is author of "Java Portlets 101" book and more than 25 articles and has a popular blog about portlet development and administration (http://wpcertification.blogspot.com)

http://www.webspherenotes.com

Letzte Blogeinträge

  • How to drain/delete/expire existing messages in Kafka

    Mittwoch, 8. November 2017

    Some time you might have a bad record in Kafka topic that you want to delete. Kafka does not provide direct option to delete specific record. Only way to delete records is to expire them. You can achieve this by setting data retention to say 1 second that expires all the old messages. You can follow these steps

  • Kafka how to reset number of partitions in a topics

    Freitag, 3. November 2017

    I wanted to figure out how to reset number of partitions in my topic in Kafka and I followed these steps

    1. I did create a sample topic called my-topic with single partition 

    spatil$ bin/kafka-topics.sh --create --zookeeper localhost:2181 --topic my-topic --replication-factor 1 --partitions 1Created topic "my-topic".
  • I used describe command to verify that my topic has actually single partition
    spatil$ bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic my-topic

  • How to reset consumer group offset

    Freitag, 3. November 2017

    First run describe on topic to check what it the current LAG its zero in this case 

    bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group user.kafkaconsumer

    Now run following command to just preview what will be the next offset if you reset 

    bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group user.kafkaconsumer --reset-offsets --to-earliest --all-topics

  • Spark program to read data from RDBMS

    Donnerstag, 12. Januar 2017

    I wanted to figure out how to connect to RDBMS from spark and extract data, so i followed these steps. You can download this project form github First i did create Address table in my local mysql like this 

    CREATE TABLE `address` (  `addressid` int(11) NOT NULL AUTO_INCREMENT,  `contactid` int(11) DEFAULT NULL,  `line1` varchar(300) NOT NULL,  `city` varchar(50) NOT NULL,  `state` varchar(50) NOT NULL,  `zip` varchar(50) NOT NULL,

  • How to implement cache (LRU Cache) using LinkedHashMap in java

    Donnerstag, 12. Januar 2017

    Recently i wanted to implement a simple Least recently used (LRU) cache in one my applications. But my use case is simple enough that instead of going for something ehcache i decided to build it on own by using java.util.LinkedHashMap As you can see from the code its very simple.

  • Spark Streaming Kafka 10 API Word Count application Scala

    Donnerstag, 12. Januar 2017

    In Spark Kafka Streaming Java program Word Count using Kafka 0.10 API blog entry i talked about how you create a simple java program that uses Spark Streaming's Kafka10 API using Java. This blog entry does the same thing but using Scala.

  • Spark Kafka Streaming Java program Word Count using Kafka 0.10 API

    Donnerstag, 12. Januar 2017

    Kafka API went through a lot of changes starting Kafka 0.9. Spark Kafka Streaming API also was changed to better support Kafka 0.9. i wanted to try that out so i built this simple Word Count application using Kafka 0.10 API. This blog entry does the same thing but using Scala.

  • How to use ElasticSearch as storage from Hive in cloudera

    Sonntag, 1. Januar 2017

    .In the Using ElasticSearch as external data store with apache hive entry i talked about how you can create a table in Hive so that actual data is stored in ElasticSearch. Problem with that approach was that i had to pass the full path to elasticsearch-hadoop-hive-<eshadoopversion>.jar as parameter every time. 

    hive -hiveconf hive.aux.jars.path=/opt/elastic/elasticsearch-hadoop-2.4.3/dist/elasticsearch-hadoop-hive-2.4.3.jar;

  • Installing ElasticSearch on existing docker container

    Samstag, 31. Dezember 2016

    I was using a Cloudera Quickstart docker image for one experiment and wanted to install ElasticSearch on it but i had trouble in accessing from my host, but i found workaround by following these steps

    • First i installed ElasticSearch by downloading and unzipping ElasticSearch version 2.4.3 and unzipping it in /opt/elastic folder
    • Then i started elasticsearch by executing /bin/elasticsearch, and it started ok.

  • Sending and Receiving JSON messages in Kafka

    Montag, 26. Dezember 2016

    Sometime back i wrote couple of articles for Java World about Kafka Big data messaging with Kafka, Part 1 and Big data messaging with Kafka, Part 2, you can find basic Producer and Consumer for Kafka along with some basic samples. I wanted to figure out how do i pass JSON message using Kafka.

    • mehr lesen