When we create a java binary with bazel, we can run it using a command like this:

1
bazel run :main

Sometimes an application requires very specific JVM flags to run correctly (For example: -Xmx:512m). These can be set like this:

1
bazel run :main --jvmopt="-Xmx:512m" 

If we need to set more than one flag, we use this syntax:

1
bazel run :main --jvmopt="-Xmx:512m" --jvmopt="-Xms:256m"

There are times that we want to figure out what command line options where used to start a java process. There are a few ways we can do this. The examples below assume PID is set to the process id we want to inspect:

ps

1
2
3
4
ps -f $PID

UID          PID    PPID  C STIME TTY      STAT   TIME CMD
adrian    316721  313854  0 16:15 pts/6    Sl+    0:00 /home/adrian/.cache/bazel/_bazel_adrian/28381a26654a75034a8803698f5ef496/execroot/__main__/bazel-out/k8-fastbuild/bin/main.runfiles/local_jdk/bin/java -classpath main.jar -Xdebug example.Main

Using jps

jps doesn’t allow us to list a single process id, but we can see the command line arguments for all java processes:

We can use ps to get all the processes running in a host. The problem is that all java processes just say java by default. For example:

1
2
3
4
5
6
ps -e | grep java

 309158 ?        00:00:16 java
 309527 ?        00:00:05 java
 313028 pts/3    00:00:00 java
 313346 ?        00:00:03 java

We can use the -f modifier to do full-format listing, which will print all the command line arguments used to start the process:

1
2
3
4
5
6
7
ps -fe | grep java

adrian    309158   87625  0 15:03 ?        00:00:16 bazel(getting-pid-for-java-process) -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/adrian/.cache/bazel/_bazel_adrian/2cff94953bdedc4a1c6a41ce83e93da8 -Xverify:none -Djava.util.logging.config.file=/home/adrian/.cache/bazel/_bazel_adrian/2cff94953bdedc4a1c6a41ce83e93da8/javalog.properties -Dcom.google.devtools.build.lib.util.LogHandlerQuerier.class=com.google.devtools.build.lib.util.SimpleLogHandler$HandlerQuerier -XX:-MaxFDLimit -Djava.library.path=/usr/share/bazel/ -Dfile.encoding=ISO-8859-1 -jar /usr/share/bazel/A-server.jar --max_idle_secs=10800 --noshutdown_on_low_sys_mem --connect_timeout_secs=30 --output_user_root=/home/adrian/.cache/bazel/_bazel_adrian --install_base=/usr/share/bazel --install_md5=754af2f3778d588aa4604927527eee4e --output_base=/home/adrian/.cache/bazel/_bazel_adrian/2cff94953bdedc4a1c6a41ce83e93da8 --workspace_directory=/home/adrian/repos/ncona-code-samples/getting-pid-for-java-process --default_system_javabase=/usr/lib/jvm/java-11-openjdk-amd64 --failure_detail_out=/home/adrian/.cache/bazel/_bazel_adrian/2cff94953bdedc4a1c6a41ce83e93da8/failure_detail.rawproto --deep_execroot --expand_configs_in_place --idle_server_tasks --write_command_log --nowatchfs --nofatal_event_bus_exceptions --nowindows_enable_symlinks --client_debug=false --product_name=Bazel --noincompatible_enable_execution_transition --option_sources=install_Ubase:/etc/bazel/bazelrc:trust_Uinstall_Ubase:/etc/bazel/bazelrc
adrian    309527  309158  0 15:03 ?        00:00:05 /home/adrian/.cache/bazel/_bazel_adrian/2cff94953bdedc4a1c6a41ce83e93da8/execroot/__main__/external/local_jdk/bin/java -XX:+UseParallelOldGC -XX:-CompactStrings --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED --patch-module=java.compiler=external/remote_java_tools_linux/java_tools/java_compiler.jar --patch-module=jdk.compiler=external/remote_java_tools_linux/java_tools/jdk_compiler.jar --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED -jar external/remote_java_tools_linux/java_tools/JavaBuilder_deploy.jar --persistent_worker
adrian    313028  311741  0 15:35 pts/3    00:00:00 /home/adrian/.cache/bazel/_bazel_adrian/2cff94953bdedc4a1c6a41ce83e93da8/execroot/__main__/bazel-out/k8-fastbuild/bin/main.runfiles/local_jdk/bin/java -classpath main.jar example.Main
adrian    313346  309158  5 15:35 ?        00:00:02 /home/adrian/.cache/bazel/_bazel_adrian/2cff94953bdedc4a1c6a41ce83e93da8/execroot/__main__/external/local_jdk/bin/java -XX:+UseParallelOldGC -XX:-CompactStrings --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED --patch-module=java.compiler=external/remote_java_tools_linux/java_tools/java_compiler.jar --patch-module=jdk.compiler=external/remote_java_tools_linux/java_tools/jdk_compiler.jar --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED -jar external/remote_java_tools_linux/java_tools/JavaBuilder_deploy.jar --persistent_worker
adrian    313431  312937  0 15:36 pts/8    00:00:00 grep --color=auto java

This is better, but there is a lot of information so it is a little hard to find the process we are looking for.

Kafka is advertised as a distributed event store and stream processing platform. It’s used in many companies to reliably pass information between diferent systems in ways that traditional databases were not designed to do.

Let’s start by understanding why Kafka is called an event store.

In traditional databases (MySQL, for example) we can store records inside tables. Running these queries:

1
2
3
INSERT INTO people(id, name) VALUES(111, "Carlos");
INSERT INTO people(id, name) VALUES(222, "Mario");
INSERT INTO people(id, name) VALUES(333, "Jose");

Gives us something like this:

You might be interested in these articles:

• Kubernetes networking with Istio
• Introduction to Jaeger Tracing

Envoy Tracing

In an Istio cluster, all applications are deployed with an Envoy proxy attached to them and Envoy supports integration with different tracing systems, this means, we can use Envoy’s features to automate tracing in our cluster.

This is what envoy can do for us:

• Initiate a trace when a request is received
• Propagate the trace-id over the cluster so spans can be connected
• Send information to tracing services like Jaeger

In this post we are going to learn how to generate a GPG key without having to answer prompts so it can be added to a script if desired.

We start by creating a file where we’ll write the details for our key. This will be the content of the file:

1
2
3
4
5
6
7
8
9
10
11
%echo Generating GPG key
Key-Type: default
Key-Type: RSA
Key-Length: 3072
Subkey-Type: RSA
Subkey-Length: 3072
Name-Real: Carlos Sanchez
Name-Email: carlos@sanchez.mex
%no-protection
%commit
%echo done

And then we can use this command to generate the key:

1
gpg --batch --gen-key <file path>

We can find all the options for generating the key in unattended key generation documentation.

Jaeger is an Open Source tracing framework written in Golang. Jaeger is compatible with OpenTracing which is an open source standard that specifies how tracing data should be formed.

Tracing

Tracing is concept that became popular with the rise of microservices (Service Oriented Architecture). When we work with microservices, the number of requests going through our internal network increases very quickly. To do its job a service might have to call a lot of services and some of those services might need to call other services.

When we make so many network requests it’s sometimes hard to understand where a failure is coming from. Tracing keeps track of the dependencies between different services and allows us to visualize it in a easy to understand manner.

In my last article we learned how to get started with Elasticsearch. In this article we are going to learn some strategies for dealing with related documents.

Relationships

Relationships between documents depend on the type of data we are storing. Some examples of relationships:

• Restaurants and locations, where a restaurant can have multiple locations, but a location can only belong to one restaurant. This is a one to many relationship.
• Orders and users, where each order belongs to a user. A user can have multiple orders but an order belongs to a single user. Also, one to many.
• Movies and actors. A movie can star multiple actors and an actor can star in multiple movies. Many to many relationship.

Why do I need a search engine?

Search is everywhere. We use Google to find websites, we search for products in Amazon, we use keywords to find videos on Youtube, etc.

From the consumer side, it’s a great way to get the information we need quickly. From the producer side, it means that we need to make sure we provide this interface the users have come to expect.

Relational or document databases allow us to create indices and find pieces of information based on ids, but they don’t work for searching for various keywords inside a text or when there are typos or synonyms involved. This is where search engines become useful; when we need to index unstructured data and offer flexible search capabilities.

In my previous article we learned how to use the fabric8 java kubernetes client. In this article we are going to explore their Informer API.

The Informer API can be used to monitor Kubernetes resources (pods, services, etc.). This can be useful for a number of things like performing actions when a resource is created, destroyed, etc.

Using Informers

Informers are used to monitor Kubernetes resources. We create an informer with SharedInformerFactory: