GitHub: me too! Port 22 doesn't work! Bummer! Corporate firewalls suck!
I find it complicated enough to establish a tunnel to GitHub through a corporate firewall to share my results. The following steps describe what I have done on my Windows 7 to finally succeed.
Install
Install cygwin with git, ssh and corkscrew.
All commands are now entered in the cygwin shell, not the Windows command shell.
We use git from the cygwin package, not from the git bash.
Configure the proxy for git
make sure you have the HTTP_PROXY environment variable set:
echo $HTTP_PROXY
should print your proxy host and port
Now enter
git config --global http.proxy $HTTP_PROXY
On to the fun part: tunnel SSH through the https port 443
Follow the ssh key generation process as described in the GitHub installation page.
The command
ssh -T git@github.com
fails because port 22 is blocked. That's why you are reading this page after all.
Create the file /.ssh/config and put the following content in it
Host gitproxy
User git
HostName ssh.github.com
Port 443
ProxyCommand /usr/bin/corkscrew.exe proxyhost proxyport %h %p
IdentityFile /.ssh/id_rsa
If you don't like the host name gitproxy, feel free to choose your own.
The indentity file is you private key file, which has been generated with ssh-keygen.
Make sure the rights are set correctly, otherwise ssh wil complain
chmod 644 /.ssh/config
Now try
ssh gitproxy
ssh will ask you for the passphrase you have defined for your ssh key
GitHub says:
Hi (your GitHub name)! You've successfully authenticated, but GitHub does not provide shell access.
Use the host gitproxy instead of github.com for all further git commands. The passphrase is prompted for the git push command and likely a bunch of others. I am new to git and GitHub, so forgive me for my lack of precision.
The programs ssh-agent and ssh-add can automate the passphrase so that you don't have to enter it every time.
Set up ssh-agent
Add
eval `ssh-agent`
to your .bashrc
Reopen the cygwin shell and run
ssh-add /.ssh/id_rsa
I am sure I have forgotten something, but hopefully this will take you 95% ;-)
Did I already mention that I hate network plumbing with a passion?
Wednesday, March 28, 2012
Tuesday, March 27, 2012
Geographic Visualization of my Twitter Graph
(Source at https://github.com/alpengeist/Twitter-Crawler)
I am experimenting with the Neo4j graph database and as an exercise I loaded the friend graph from Twitter, starting with me (@alpengeist_de), in depth 4 and width 40 into the DB. I then enhanced the data with the geo location information from Yahoo Places. This process takes a while, because the Twitter REST API limits to 150 requests per hour for non-registered applications. Behind a corporate proxy it is even worse, because the tracking is on the IP address and my colleagues consume the quota as well.
I am experimenting with the Neo4j graph database and as an exercise I loaded the friend graph from Twitter, starting with me (@alpengeist_de), in depth 4 and width 40 into the DB. I then enhanced the data with the geo location information from Yahoo Places. This process takes a while, because the Twitter REST API limits to 150 requests per hour for non-registered applications. Behind a corporate proxy it is even worse, because the tracking is on the IP address and my colleagues consume the quota as well.
My Java program caches the friend data in a simple CSV file, so I can start it anytime and get another bunch of users from Twitter. The geodata is cached in a properties file (the simplest K/V store there is :-)
I can quickly generate the Neo4j graph database with data from disk.
I have collected about 30000 users so far. About 18000 of them have the full data set. For visualization I started with the open source tool Gephi, whose Geo Layouter I have used to produce the images.
I did a node ranking by follower count to fatten the dots a little, and a color partitioning by country. Yahoo Places returns a quality measure, so I filtered on that as well. Et voilà, the continents emerge.
Not everyone in Twitter has entered a real location. Many say they are in "the internet", which is located in Brazil according to Yahoo. It seems like the Internet is enjoying itself in a pleasant climate and dancing Samba! Another good one is @artcika, who claims to be "in the middle of the map". Yahoo locates that in Papua Neuginea.
Those errors are not easy to filter out.
Not everyone in Twitter has entered a real location. Many say they are in "the internet", which is located in Brazil according to Yahoo. It seems like the Internet is enjoying itself in a pleasant climate and dancing Samba! Another good one is @artcika, who claims to be "in the middle of the map". Yahoo locates that in Papua Neuginea.
Those errors are not easy to filter out.
 |
| Twitter Graph layouted with Gephi Geo Layouter (Mercator projection), partitioned by country |
The little spots outside the continents are mostly due to nonsense location data from Twitter where Yahoo Places was still confident to have delivered something useful. However, there are actually users in Alaska, Hawaii, and Iceland :-)
I have plans to get familiar with the D3 JavaScript library for visualization, but that will take some effort.
Finally, here is a version with the connections switched on. There is not much information one can draw from this. I just enjoy the emerging color patterns.
Finally, here is a version with the connections switched on. There is not much information one can draw from this. I just enjoy the emerging color patterns.
Monday, March 12, 2012
Even your Java VM is eventually consistent - or worse
When studying distributed systems, or in my case more specifically distributed databases, you encounter the term eventual consistency and the CAP theorem.
A multicore processor system is actually also a kind of distributed system. There is no unreliable network communication in between the cores, however it has nodes (cores) with isolated caches and registers, which are in fact distributed state like a distributed DB has.
The book Java Concurrency In Practice by Brian Goetz et al deals with Java concurrency in an almost overwhelming depth. In chapter 3.1 "Visibility" I found an interesting analogy to eventually consistent databases.
The story is that variables in a multithreaded program are not consistent among threads unless the access is synchronized. With unsynchronized variables the compiler may reorder statements and cache values in registers or processor-specific caches, so that the individual threads have inconsistent state.
With synchronization, we trade in a little of Availability of the CAP triangle to achieve Consistency. I won't go too far with this analogy, since Partition Tolerance has no place in a multicore one chip processor. Nevertheless, it is the same thing as in distributed systems.
I dare to publish their example here without permission, not without recommending the book as the standard literature for the subject, of course :-)
A JVM can even be worse than eventually consistent: never consistent!
public class NoVisibility {
private static boolean ready;
private static int number;
private static class ReaderThread extends Thread {
public void run() {
while (!ready)
Thread.yield();
System.out.println(number);
}
}
public static void main(String[] args) {
new ReaderThread().start();
number = 42;
ready = true;
}
}
In this example, the ReaderThread may never finish because the boolean ready flag may never be true, even if it is certainly true in the main thread. Also, if it was true for the ReaderThread, there is no guarantee that number is 42. It may as well be 0 because of statement reordering effects.
As the authors point out, this code is as simple as it gets and boom - disaster and terribly hard to find errors are waiting to give you a hard time.
A less strong alternative to synchronization is declaring a variable as volatile. This instructs the compiler to avoid caching in registers. The value is written to and fetched from memory, which is shared, consistent state for all threads.
Personally, I don't think that mastering concurrency is a subject, which the majority of Java developers will be capable of down to the finest details. It is even more difficult than generics, another dark corner for many. Serious concurrent programming in Java requires dropping the familiar standard data structures from java.util in favor of java.util.concurrent, which is presented in detail in the book. The language itself does not hide any complexity, it hits you in the face brutally. Future will show whether most use cases can be satisfied with java.util.concurrent (together with Java 8 lambdas maybe) or if Java concurrency becomes a bone breaker.
A multicore processor system is actually also a kind of distributed system. There is no unreliable network communication in between the cores, however it has nodes (cores) with isolated caches and registers, which are in fact distributed state like a distributed DB has.
The book Java Concurrency In Practice by Brian Goetz et al deals with Java concurrency in an almost overwhelming depth. In chapter 3.1 "Visibility" I found an interesting analogy to eventually consistent databases.
The story is that variables in a multithreaded program are not consistent among threads unless the access is synchronized. With unsynchronized variables the compiler may reorder statements and cache values in registers or processor-specific caches, so that the individual threads have inconsistent state.
With synchronization, we trade in a little of Availability of the CAP triangle to achieve Consistency. I won't go too far with this analogy, since Partition Tolerance has no place in a multicore one chip processor. Nevertheless, it is the same thing as in distributed systems.
I dare to publish their example here without permission, not without recommending the book as the standard literature for the subject, of course :-)
A JVM can even be worse than eventually consistent: never consistent!
public class NoVisibility {
private static boolean ready;
private static int number;
private static class ReaderThread extends Thread {
public void run() {
while (!ready)
Thread.yield();
System.out.println(number);
}
}
public static void main(String[] args) {
new ReaderThread().start();
number = 42;
ready = true;
}
}
In this example, the ReaderThread may never finish because the boolean ready flag may never be true, even if it is certainly true in the main thread. Also, if it was true for the ReaderThread, there is no guarantee that number is 42. It may as well be 0 because of statement reordering effects.
As the authors point out, this code is as simple as it gets and boom - disaster and terribly hard to find errors are waiting to give you a hard time.
A less strong alternative to synchronization is declaring a variable as volatile. This instructs the compiler to avoid caching in registers. The value is written to and fetched from memory, which is shared, consistent state for all threads.
Personally, I don't think that mastering concurrency is a subject, which the majority of Java developers will be capable of down to the finest details. It is even more difficult than generics, another dark corner for many. Serious concurrent programming in Java requires dropping the familiar standard data structures from java.util in favor of java.util.concurrent, which is presented in detail in the book. The language itself does not hide any complexity, it hits you in the face brutally. Future will show whether most use cases can be satisfied with java.util.concurrent (together with Java 8 lambdas maybe) or if Java concurrency becomes a bone breaker.
Tuesday, November 1, 2011
My Four Natural Laws Of Programming
(work in progress...)
I wrote my first program in BASIC on a Commodore 8032 in 1984. The computer was magical and I had no idea what was actually going on inside it.
Today the magic of the machine is gone. Yet I still feel the magic of the structure and aesthetics of programming.
Recently, I realized that the more I program, the fewer rules I use do decide what's good.
There are loads of rules out there describing what good code looks like. Too many of them. Put yourself into the mind of a beginner and try to figure how to learn them. I try to condense my rules to a small universal set, which include many others as special cases. I like to draw an analogy to the fundamental forces of physics, which became less numerous with time, embracing more forces previously seen as individual ones.
Further, I believe in the emergence of software architecture when the laws are applied continuously and code gets refactored accordingly.
Do you also think "I know good code when I see it"? I definitely do. Programming is a very visual activity. Good code always looks good.
When I do code reviews, I find myself applying a handful of universal laws, which reliably uncover flaws without having to know what the code is actually doing. The rest if covered by sufficient testing.
I'll write down my personal laws of programming to share them and to become aware of them myself. They lean towards object-oriented programming in some areas because that is what I do most. I also talk about how I check the laws during a code review.
#1 Redundancy
Redundancy means two things: repetition and waste.
Repeating code is an indicator for missing abstraction. It can appear everywhere, from data access patterns, which repeatedly follow the same chains, to algorithmic patterns, which differ in little details not properly abstracted.
Waste code adds no value, solves no relevant problem. However, it introduces bugs and maintenance cost. Waste conjures up workarounds due to missing understanding by programmers. Useless layering and over-abstraction fall into this category.
Using code generators to produce waste makes them waste generators.
The art is to find the right abstraction balance to avoid repetition without introducing waste.
Automated code duplication checkers help to find repetition. Waste can only be identified in the context of the given architecture constraints. Waste in one case may be a necessary construct somewhere else. Mechanical exercise of too limited rules is harmful here. This is where experience makes all the difference.
#2 Consistency
Consistency is about recurring patterns, which do not contradict each other.
Consistent code becomes familiar quickly. It has no exceptions to rules.
Consistency has many facets. Names are chosen consistently. An API uses a consistent level of abstraction. Data access patterns are similar. Things are identified consistently and not sometimes by IDs, sometimes as object refs, for example. A class hierarchy is not designed only to be torn apart with
Error handling is another large field for consistency hazards.
Checking consistency is something no machine can do for you (yet). As a code reader, I critically watch out for surprises, which introduce a change in patterns.
#3 Fat
With fat I am not referring to code, which has no use and is redundant. Fat is missing structure, it fails to provide a proper segmentation of responsibility. Too much is packed into one location.
Fat is easy to see. In methods for example, there are big code blocks or unrelated blocks. A method starts with one task and continues with some other and you cannot quickly assess where and why. If the programmer was diligent, he left some comments between the blocks. Classes contain too many methods and again you cannot see where the boundaries are between the tasks a class has.
You can tell by one look that something is not ok, regardless what the code actually does. It is purely a matter of form. Automated metrics like XS (excessive structural complexity) in Structure101 or the well-known cyclomatic complexity help to find the hot spots quickly.
#4 Coupling
Coupling addresses the problem of who knows what.
This is a huge field and I think the most important aspect of architecture on large and small scales.
The fundamental question is:
Why do I access this information here and now from that source?
Bad coupling restricts the source of the information in its ability to change. It is like pinning a tense wire to it. You find coupling problems in interfaces to external systems, which are too tight to be backwards compatible, in missing encapsulation, in badly organized separation of concerns causing wild dependencies (hair balls), in methods juggling with too many things at once, in packages with little coherence. Bad coupling produces dependency cycles between packages.
Coupling also comes in multiple dimensions,
being the most important ones.
There are many techniques to reduce coupling in specific situations. Knowing them, and as with abstractions, using them wisely, is a matter of experience.
Identifying cycles in non-trivial programs is impossible manually. For Java, Structure101 became an indispensable tool for me. I seem to be too dumb or lazy to understand a colored dependency matrix like IntelliJ or Sonar have to offer. You need the help of the machine here.
That's it. Really. The facets are numerous and need to be learned in each environment. Still, I believe it all comes down to a handful of principles. When I struggle with some code or design, I can always root the problem in the four laws. And every once in a while, it was my own fault :-)
Programming as technique is not magical, it is a craft. Sometimes we forget and make a big fuss about it ("Popanz und Gedöns" in German). The magic shines when stuff is done right, like in a painting of a master.
I wrote my first program in BASIC on a Commodore 8032 in 1984. The computer was magical and I had no idea what was actually going on inside it.
Today the magic of the machine is gone. Yet I still feel the magic of the structure and aesthetics of programming.
Recently, I realized that the more I program, the fewer rules I use do decide what's good.
There are loads of rules out there describing what good code looks like. Too many of them. Put yourself into the mind of a beginner and try to figure how to learn them. I try to condense my rules to a small universal set, which include many others as special cases. I like to draw an analogy to the fundamental forces of physics, which became less numerous with time, embracing more forces previously seen as individual ones.
Further, I believe in the emergence of software architecture when the laws are applied continuously and code gets refactored accordingly.
Do you also think "I know good code when I see it"? I definitely do. Programming is a very visual activity. Good code always looks good.
When I do code reviews, I find myself applying a handful of universal laws, which reliably uncover flaws without having to know what the code is actually doing. The rest if covered by sufficient testing.
I'll write down my personal laws of programming to share them and to become aware of them myself. They lean towards object-oriented programming in some areas because that is what I do most. I also talk about how I check the laws during a code review.
#1 Redundancy
Redundancy means two things: repetition and waste.
Repeating code is an indicator for missing abstraction. It can appear everywhere, from data access patterns, which repeatedly follow the same chains, to algorithmic patterns, which differ in little details not properly abstracted.
Waste code adds no value, solves no relevant problem. However, it introduces bugs and maintenance cost. Waste conjures up workarounds due to missing understanding by programmers. Useless layering and over-abstraction fall into this category.
Using code generators to produce waste makes them waste generators.
The art is to find the right abstraction balance to avoid repetition without introducing waste.
Automated code duplication checkers help to find repetition. Waste can only be identified in the context of the given architecture constraints. Waste in one case may be a necessary construct somewhere else. Mechanical exercise of too limited rules is harmful here. This is where experience makes all the difference.
#2 Consistency
Consistency is about recurring patterns, which do not contradict each other.
Consistent code becomes familiar quickly. It has no exceptions to rules.
Consistency has many facets. Names are chosen consistently. An API uses a consistent level of abstraction. Data access patterns are similar. Things are identified consistently and not sometimes by IDs, sometimes as object refs, for example. A class hierarchy is not designed only to be torn apart with
instanceof later. Consistency shows in method signatures in the order of parameters, and imperative vs. functional style. Countless examples are possible here.Error handling is another large field for consistency hazards.
Checking consistency is something no machine can do for you (yet). As a code reader, I critically watch out for surprises, which introduce a change in patterns.
#3 Fat
With fat I am not referring to code, which has no use and is redundant. Fat is missing structure, it fails to provide a proper segmentation of responsibility. Too much is packed into one location.
Fat is easy to see. In methods for example, there are big code blocks or unrelated blocks. A method starts with one task and continues with some other and you cannot quickly assess where and why. If the programmer was diligent, he left some comments between the blocks. Classes contain too many methods and again you cannot see where the boundaries are between the tasks a class has.
You can tell by one look that something is not ok, regardless what the code actually does. It is purely a matter of form. Automated metrics like XS (excessive structural complexity) in Structure101 or the well-known cyclomatic complexity help to find the hot spots quickly.
#4 Coupling
Coupling addresses the problem of who knows what.
This is a huge field and I think the most important aspect of architecture on large and small scales.
The fundamental question is:
Why do I access this information here and now from that source?
Bad coupling restricts the source of the information in its ability to change. It is like pinning a tense wire to it. You find coupling problems in interfaces to external systems, which are too tight to be backwards compatible, in missing encapsulation, in badly organized separation of concerns causing wild dependencies (hair balls), in methods juggling with too many things at once, in packages with little coherence. Bad coupling produces dependency cycles between packages.
Coupling also comes in multiple dimensions,
- Structure
- Time
- Location
being the most important ones.
There are many techniques to reduce coupling in specific situations. Knowing them, and as with abstractions, using them wisely, is a matter of experience.
Identifying cycles in non-trivial programs is impossible manually. For Java, Structure101 became an indispensable tool for me. I seem to be too dumb or lazy to understand a colored dependency matrix like IntelliJ or Sonar have to offer. You need the help of the machine here.
That's it. Really. The facets are numerous and need to be learned in each environment. Still, I believe it all comes down to a handful of principles. When I struggle with some code or design, I can always root the problem in the four laws. And every once in a while, it was my own fault :-)
Programming as technique is not magical, it is a craft. Sometimes we forget and make a big fuss about it ("Popanz und Gedöns" in German). The magic shines when stuff is done right, like in a painting of a master.
Friday, October 21, 2011
CDI Events solve fancy problems elegantly (Part 2)
Problem: Does your application have a heartbeat?
I used to spread the phrase "every decent application needs a timer bean". Since JEE6, I have rephrased it to "every decent application needs a heart beat".
Whaddaya?
The @Schedule annotation improves upon the old timer service because it has zero admin effort. However, it has issues:
The configuration of @Schedule is in the static code, you cannot modify it at runtime, for example with values from a configuration file outside the deployment package.
Further, I think most timed routine jobs in an application are of the simple sort every minute, every quarter hour, every hour, at time x every day. I rarely find a reason to be more granular than this.
@Scheduled has been inspired by cron, which is flexibly configured with a text file. This is way different to compiling the static config into a class file in an EAR and not being able to modify it in the deployed application. That's very "un-cronish".
How about this idea: a heartbeat event is fired every minute and subscribers can observe it. The event has inspection methods to find out about the event time properties. The observer can be configured at runtime using data from an external config file.
The HeartbeatEvent has a bunch of inquiry methods. isTime(), isMinuteInterval, isHourInterval() take parameters, which can be fetched from a configuration, for example. Having the Calendar at hand, it is possible to invent all kinds of inquiries. Those are the ones that I find most useful.
The HeartbeatEmitter:
Example heartbeat observer:
I used to spread the phrase "every decent application needs a timer bean". Since JEE6, I have rephrased it to "every decent application needs a heart beat".
Whaddaya?
The @Schedule annotation improves upon the old timer service because it has zero admin effort. However, it has issues:
The configuration of @Schedule is in the static code, you cannot modify it at runtime, for example with values from a configuration file outside the deployment package.
Further, I think most timed routine jobs in an application are of the simple sort every minute, every quarter hour, every hour, at time x every day. I rarely find a reason to be more granular than this.
@Scheduled has been inspired by cron, which is flexibly configured with a text file. This is way different to compiling the static config into a class file in an EAR and not being able to modify it in the deployed application. That's very "un-cronish".
How about this idea: a heartbeat event is fired every minute and subscribers can observe it. The event has inspection methods to find out about the event time properties. The observer can be configured at runtime using data from an external config file.
The HeartbeatEvent has a bunch of inquiry methods. isTime(), isMinuteInterval, isHourInterval() take parameters, which can be fetched from a configuration, for example. Having the Calendar at hand, it is possible to invent all kinds of inquiries. Those are the ones that I find most useful.
public class HeartbeatEvent {
private Calendar calendar;
public HeartbeatEvent() {
this.calendar = new GregorianCalendar();
}
/**
* "at x hour and y minutes"
* Use minute == 0 to test against a specific full hour.
* @param hour 0-23
* @param minute 0-59
* @return true if match
*/
public boolean isTime(int hour, int minute) {
return calendar.get(Calendar.HOUR_OF_DAY) == hour && calendar.get(Calendar.MINUTE) == minute;
}
/**
* "every n minutes"
* @param minutes minute interval 0-59
* @return true if match
*/
public boolean isMinuteInterval(int minutes) {
return calendar.get(Calendar.MINUTE) % minutes == 0;
}
/**
* "every n hours"
* @param hour 0-23
* @return true if match
*/
public boolean isHourInterval(int hour) {
return calendar.get(Calendar.HOUR_OF_DAY) % hour == 0 && isFullHour();
}
public boolean isFullHour() {
return calendar.get(Calendar.MINUTE) == 0;
}
public boolean isHalfHour() {
return isMinuteInterval(30);
}
public boolean isQuarterHour() {
return isMinuteInterval(15);
}
}
@Singleton
public class HeartbeatEmitter {
@Inject
private Event<HeartbeatEvent> heartbeat;
@Schedule(hour="*", minute="*", persistent = false)
public void emit() {
heartbeat.fire(new HeartbeatEvent());
}
}
public void fileCleanup(@Observes HeartbeatEvent heartbeat) {
if (heartbeat.isFullHour()) {
// do file cleanup every full hour
}
}
CDI Events solve fancy problems elegantly (Part 1)
Events are a sweet spot in CDI. I have done a lot of enterprise application integration (EAI) with queuing systems and I am happy to find that message passing (aka events) has arrived at JEE6 in a nicely integrated fashion.
Problem: Arbitrary-time-delayed execution of operations
Imagine an application where user A triggers an operation, which needs the confirmation of another user B before it can be executed. The point of time when B decides to confirm is not known in advance and arbitrary.
Possible scenario: the request for confirmation is sent from A to B via the internal notification system of the application. B opens the notification, clicks on the "Confirm" button, and executes the operation on behalf of A.
"I need a workflow system!" some enterprise architects blurb immediately.
"I need Lotus Notes!" ...no comment :-)
No you don't. With CDI it is surprisingly little code.
With CDI you can go about like this:
From an architectural view, there are some nice aspects to mention:
To define an Event in CDI you need two things:
public class HelloEvent {
public String greetings;
public HelloEvent(String greetings) //... as usual
// to hide the preferred (de)serialization format
// the event has two methods to take care of that.
// XML or JSON are possible choices.
public String toString() // ...
public static HelloEvent fromString(String s) //...
}
Designing the Hello event service
The event producer must be injected into a class. An event service is a good place. It's job is to encapsulate the event mechanism.
@Inject private Event<HelloEvent> helloEventProducer;
To allow clients to fire a HelloEvent without knowing the plumbing, the event service exposes a method like this:
public void sendHello(HelloEvent event) {
helloEventProducer.fire(event);
}
The observer listens to HelloEvents. It also lives in the event service. This keeps all the event business in one place.
public void receiveHello(@Observes HelloEvent event) {
// How the hello greetings are actually processed
// is not important here. Some imaginary HelloProcessor
// is responsible for that. It does not know about the
// Event business and only cares for the greeting text.
helloProcessor(event.greetings);
}
I needed a transactional event service, so I chose a stateless EJB:
@Stateless
public class HelloEventService {
@Inject private Event<HelloEvent> helloEventProducer;
public void sendHello(HelloEvent event) //...
public void receiveHello(@Observes HelloEvent event) //...
}
We are done plumbing the CDI events.
Designing the Hello confirmation service
Next is a rough sketch of a HelloService, which encapsulates the time-delayed confirmation process.
@Stateless
public class HelloService {
@EJB
private HelloEventService eventSvc;
public void sendHelloConfirmed(String greeting) {
// Extremely simplified: store the event
// serialized for later retrieval.
storeSomewhere(new HelloEvent(greeting).toString());
}
public void executeHello(String serializedEvent) {
eventSvc.sendHello(HelloEvent.fromString(serializedEvent));
}
}
The HelloService has no dependency to CDI event handling, nor has it any clue about the event (de)serialization format. It only knows the HelloEvent and the HelloEventService.
How the serialized event string is passed around is not relevant for the concept. I think you get the idea. To wrap up, let's look at the client code, which is executed by user A (the sender) and user B (the guy who confirms A's intended operation).
The client code
Code executed by user A:
@EJB HelloService hello;
...
hello.sendHelloConfirmed("My goodness, you became fat!");
User B code executed at some later time:
@EJB HelloService hello;
...
String serializedEvent = ... // comes from some storage
hello.executeHello(serializedEvent);
Neither EventService, HelloEvent, nor any CDI Event artifacts appear in the client code. The mechanism of the confirmation process is encapsulated.
The concept as such is not exclusive to CDI events, of course, and could have been realized with JMS and message-driven beans as well. However, the simple and elegant integration with CDI makes the message passing (aka event) architectural pattern easier accessible than before.
Sidenote: JSON Serialization
I like JSON for its natural mapping of composed objects, lists, arrays, and maps. As with XML, with JSON mapping libraries there is a choice between comfort and flexibility.
XStream for example can (de)serialize instances, but is sensitive to versioning issues. It dies when it hits content during deserialization, which it cannot bind. This is typical for tight language bindings and I've seen that with XML already.
json-simple is totally generic, handles any valid JSON, but you need to know how to find things in the nested Maps and Arrays, and you need to map values yourself.
I think JXPath + json-simple is a very flexible and powerful combination for nested structures.
Problem: Arbitrary-time-delayed execution of operations
Imagine an application where user A triggers an operation, which needs the confirmation of another user B before it can be executed. The point of time when B decides to confirm is not known in advance and arbitrary.
Possible scenario: the request for confirmation is sent from A to B via the internal notification system of the application. B opens the notification, clicks on the "Confirm" button, and executes the operation on behalf of A.
"I need a workflow system!" some enterprise architects blurb immediately.
"I need Lotus Notes!" ...no comment :-)
No you don't. With CDI it is surprisingly little code.
With CDI you can go about like this:
- (user A) Create an event (more on that later) and serialize it to a format, which can be stored easily. I've used JSON for that purpose.
- (user A) Transport the serialized event to user B in the application. Store it in a DB for example, where B can pick it up later.
- (user B much later) Deserialize the event and fire it.
- (system) Observe the event and carry out the operation associated with it. It behaves as if user A had started the operation directly.
From an architectural view, there are some nice aspects to mention:
- The inversion of dependency with events decouples the creator of the event from the implementation of the operation. This is analogous to putting an interface between caller and implementation
- The event observer has no knowledge about the mechanism how the event is transported from A to B, nor how the event has been created. All it knows is what event to observe and what method to call with the data from the event.
- In a publish-subscribe fashion, more observers can be added non-invasively.
- The same observer can be used without time delay when the event creator fires the event immediately, without prior serialization.
To define an Event in CDI you need two things:
- A class, which represents the event (actually the message payload) an Observer listens to.
- An Event instance, which is used to fire (publish) the event object.
public class HelloEvent {
public String greetings;
public HelloEvent(String greetings) //... as usual
// to hide the preferred (de)serialization format
// the event has two methods to take care of that.
// XML or JSON are possible choices.
public String toString() // ...
public static HelloEvent fromString(String s) //...
}
Designing the Hello event service
The event producer must be injected into a class. An event service is a good place. It's job is to encapsulate the event mechanism.
@Inject private Event<HelloEvent> helloEventProducer;
To allow clients to fire a HelloEvent without knowing the plumbing, the event service exposes a method like this:
public void sendHello(HelloEvent event) {
helloEventProducer.fire(event);
}
The observer listens to HelloEvents. It also lives in the event service. This keeps all the event business in one place.
public void receiveHello(@Observes HelloEvent event) {
// How the hello greetings are actually processed
// is not important here. Some imaginary HelloProcessor
// is responsible for that. It does not know about the
// Event business and only cares for the greeting text.
helloProcessor(event.greetings);
}
I needed a transactional event service, so I chose a stateless EJB:
@Stateless
public class HelloEventService {
@Inject private Event<HelloEvent> helloEventProducer;
public void sendHello(HelloEvent event) //...
public void receiveHello(@Observes HelloEvent event) //...
}
We are done plumbing the CDI events.
Designing the Hello confirmation service
Next is a rough sketch of a HelloService, which encapsulates the time-delayed confirmation process.
@Stateless
public class HelloService {
@EJB
private HelloEventService eventSvc;
public void sendHelloConfirmed(String greeting) {
// Extremely simplified: store the event
// serialized for later retrieval.
storeSomewhere(new HelloEvent(greeting).toString());
}
public void executeHello(String serializedEvent) {
eventSvc.sendHello(HelloEvent.fromString(serializedEvent));
}
}
The HelloService has no dependency to CDI event handling, nor has it any clue about the event (de)serialization format. It only knows the HelloEvent and the HelloEventService.
How the serialized event string is passed around is not relevant for the concept. I think you get the idea. To wrap up, let's look at the client code, which is executed by user A (the sender) and user B (the guy who confirms A's intended operation).
The client code
Code executed by user A:
@EJB HelloService hello;
...
hello.sendHelloConfirmed("My goodness, you became fat!");
User B code executed at some later time:
@EJB HelloService hello;
...
String serializedEvent = ... // comes from some storage
hello.executeHello(serializedEvent);
Neither EventService, HelloEvent, nor any CDI Event artifacts appear in the client code. The mechanism of the confirmation process is encapsulated.
The concept as such is not exclusive to CDI events, of course, and could have been realized with JMS and message-driven beans as well. However, the simple and elegant integration with CDI makes the message passing (aka event) architectural pattern easier accessible than before.
Sidenote: JSON Serialization
I like JSON for its natural mapping of composed objects, lists, arrays, and maps. As with XML, with JSON mapping libraries there is a choice between comfort and flexibility.
XStream for example can (de)serialize instances, but is sensitive to versioning issues. It dies when it hits content during deserialization, which it cannot bind. This is typical for tight language bindings and I've seen that with XML already.
json-simple is totally generic, handles any valid JSON, but you need to know how to find things in the nested Maps and Arrays, and you need to map values yourself.
I think JXPath + json-simple is a very flexible and powerful combination for nested structures.
Thursday, October 20, 2011
Stuff they don't tell you about CDI in JEE6
So, you have read the WELD documentation and watched presentations like http://www.parleys.com/#st=5&id=2668&sl=26 (recommended). The community is currently very actively spreading the concepts of CDI. I started digging into it in Nov 2010 and hit aspects of CDI in practice with Glassfish 3.0, which I have not found to be addressed clearly.
About noisy Interceptors
Interceptors allow you to wrap cross-cutting concerns - call them aspects if you prefer - around bean instances without modifying the bean implementation. Think about tracking, auditing, authorization, measuring call time, exception mapping, whatever.
EJB containers actually use this technique themselves to implement exception wrapping for example.
I first approach is to design one interceptor per aspect and stack them up at the class declaration. This may look like this for a stateless session bean:
@ExceptionMapped
@Tracked
@Audited
@Stateless
public class TheBean {
}
Good? Well, in theory. However, it fails in practice when you hit an Exception in your bean's code.
Interceptors produce a considerable amount of noise in a stacktrace. Since there are already a number of interceptors in place without your interceptor code (I observed this for EJBs in Glassfish 3), the actual exception point gets cut off in the stacktrace, which you print into a log, for example. The stacktraces become absurdly long.
You are blind!
So, what's the solution then? I am sorry, but you need to bundle some aspects together in one interceptor. That's what I did. The interceptor can still delegate to specialized classes, which take care of the aspects.
I rather have a less elegant design but see the point of an exception in my EJB's code.
About Stereotypes (or: knowing me, knowing you NOT)
Stereotypes are cool. Identify recurring annotation patterns in a design and put them together into a stereotype.
So I thought. Then I fell into the gap between EJB and CDI.
In my design, I have EJBs, which serve as the transactional entry point for JSF backing beans. They carry the interceptors (@ExceptionMapped and @Logged) and the usual EJB annotations:
@ExceptionMapped
@Logged
@Stateless
public class ServiceBean {
}
Further, I have the repository EJBs, which are internal to the service layer and cannot be called by a backing bean, because they are declared with MANDATORY transaction.
@Logged
@Stateless
@TransactionAttribute(TransactionAttributeType.MANDATORY)
public class RepositoryServiceBean {
}
Wouldn't it be much nicer to have two stereotypes containing everything:
@PublicService
public class ServiceBean {...}
@RepositoryService
public class RepositoryServiceBean {...}
So I stuffed all the annotations into the stereotypes and failed.
It kind of makes sense, when you think about it. EJB was there first and CDI has been integrated later. CDI knows EJB but not the other way round. Is this temporary and will JEE finally unite both containers? I really hope so. In the current state, it is quite confusing in some corners.
About noisy Interceptors
Interceptors allow you to wrap cross-cutting concerns - call them aspects if you prefer - around bean instances without modifying the bean implementation. Think about tracking, auditing, authorization, measuring call time, exception mapping, whatever.
EJB containers actually use this technique themselves to implement exception wrapping for example.
I first approach is to design one interceptor per aspect and stack them up at the class declaration. This may look like this for a stateless session bean:
@ExceptionMapped
@Tracked
@Audited
@Stateless
public class TheBean {
}
Good? Well, in theory. However, it fails in practice when you hit an Exception in your bean's code.
Interceptors produce a considerable amount of noise in a stacktrace. Since there are already a number of interceptors in place without your interceptor code (I observed this for EJBs in Glassfish 3), the actual exception point gets cut off in the stacktrace, which you print into a log, for example. The stacktraces become absurdly long.
You are blind!
So, what's the solution then? I am sorry, but you need to bundle some aspects together in one interceptor. That's what I did. The interceptor can still delegate to specialized classes, which take care of the aspects.
I rather have a less elegant design but see the point of an exception in my EJB's code.
About Stereotypes (or: knowing me, knowing you NOT)
Stereotypes are cool. Identify recurring annotation patterns in a design and put them together into a stereotype.
So I thought. Then I fell into the gap between EJB and CDI.
In my design, I have EJBs, which serve as the transactional entry point for JSF backing beans. They carry the interceptors (@ExceptionMapped and @Logged) and the usual EJB annotations:
@ExceptionMapped
@Logged
@Stateless
public class ServiceBean {
}
Further, I have the repository EJBs, which are internal to the service layer and cannot be called by a backing bean, because they are declared with MANDATORY transaction.
@Logged
@Stateless
@TransactionAttribute(TransactionAttributeType.MANDATORY)
public class RepositoryServiceBean {
}
Wouldn't it be much nicer to have two stereotypes containing everything:
@PublicService
public class ServiceBean {...}
@RepositoryService
public class RepositoryServiceBean {...}
So I stuffed all the annotations into the stereotypes and failed.
- The @Stateless annotation needs to stay with the class definition, otherwise the deployment of the EJB fails. From Glassfish's point of view, there simply wasn't any EJB declared to deploy.
- The @TransactionAttribute silently ceased to have effect and also needed to stay with the class.
- In both cases, WELD swallowed the nonsense without complaining.
It kind of makes sense, when you think about it. EJB was there first and CDI has been integrated later. CDI knows EJB but not the other way round. Is this temporary and will JEE finally unite both containers? I really hope so. In the current state, it is quite confusing in some corners.
Subscribe to:
Posts (Atom)