Monday, July 19, 2010

Advanced usage of Visitor-Pattern

Have you ever used the Visitor Pattern to resolve the specific type of a collection of objects? What you usually do is defining a common interface and add an accept()-method taking the visitor interface and in the implementation invoke the visit()-method from the visitor.

Here is an example for following hierarchy



The INodeElement might look like this:



and the IHeadElement linke this


Later on, in your implementation code you can use the visitor to lookup the concrete type



As you can see there is no typecast needed, so you never get a ClassCastException. However if you look at the example, one major drawback of the visitor pattern is that if the hierarchy is not matching exactly, you have a bunch of empty methods. These methods can lead to confusion and doing nothing or throwing an exception isn't much better than a ClassCastException.

If you have the control over the source, you can easily introduce a new more specific type in your hierarchy to workaround this.



Now you can tag the child elements (in this case we have only one)



and the return type of the list can be changed to a more specific one



Using the new more specific visitor, there are no more unnecessary methods



Problem solved? Yes, if you have the freedom to modify each element for tagging it, when ever you create a child list. Otherwise you should look for a better solution.

Think a short time about, what you really want to do. You want to restrict the list to adding elements only of specific types and allow to query them later, while making sure that no type is forgotten when the code changes. Another possibility to add a visitor is to wrap the list element. To do so, remove the IHeadElementChild interface and the corresponding accept-method from the ITitelElement. Now add an implemention for the IHeadElementChild interface. Create factory method for each element allow in the childlist.



In the implementation of the IHeadElement always add the wrapper to the list, everything else can be left as it is. Now you are completely independent of the class hierarchy. Note that this solution will also work with classes provided by foreign frameworks.

If you already used actifsource to describe and generate your class hierarchy, you can leave the whole work to the generator and gain the advantage of more type-safety and less casts.

Monday, July 5, 2010

Modeling Software Architecture

In my last blog On the semantics of explicit dependency relations I explained the concept of explicit dependency relations between domain-specific components. The big advantage shows up when modeling on an architectural level.

A complex system is divided in layers/tiers and packages to manage code artifacts. This technical architecture provides an overview on the implementation.

On the other hand we find the business architecture where packages, areas, functionality, epics, etc. are used to structure domain objects.

On the semantics of projects and packages
Both the technical and the business architecture have one thing in common: they are structured not only with packages but also with projects. Modern IDE's as Eclipse, MS Visual Studio or InteliJ's Idea animate developers to separate different concerns in different projects.

All artifacts of a software system are structured using projects and packages.
But architecture lays above projects. Architecture describes how a system is divided into projects. This fact leads to the problem, that architecture can never be described by its projects alone.

It seems that describing architecture needs kind of an Über-Project which contains the overview how projects are interrelated. This overview shows packages and their dependencies as shown in my blog before.

There has to be an overview how projects are interrelated. This overview is the most important artifact to understand the technical architecture or business architecture.
The tricky thing: The overview provides an entry point whereat the details are managed in the projects. But nevertheless this architectural project should contain all the information to get a complete overview. This includes also detail information contained in the particular projects.

Let's assume that all green packages are modeled in the architecture overview while the red packages are realized as projects.



  • The architectural overview seems to define packages which contain other packages
  • At a certain level the architectural overview seems to reference packages from other projects to illustrate the dependencies

Packages as resources
In the current version of actifsource packages are reflected as file system folders. But it seems obvious that packages are much more then folders.

  • Package-structures have a project-specific meta model
  • Packages declare dependencies to other packages (according the meta model)
  • Package structures may be distributed over several projects (package references)
  • Packages can be mapped to file system folders
Packages are resources that contain other resources. Modeling packages as folders is just one way. Packages might also be modeled in the architectural overview without a folder representation. And package hierarchy structures are not limited to one project but distributed over many projects.

In one of my next articles I try to examine the technicalities.

Saturday, July 3, 2010

On the semantics of explicit dependency relations

Meta-modeling leads to components built along the own-relations (compositions) and to component-interdependencies along structural use-relations.

Component interdependencies might appear deeply nested in the component. Remember that component comes from the Latin word "componere" which means being composed. The following example shows a use-relation between A.B and X.Y.

Just looking at the components without knowing the internals we can say that there is an implicit dependency between the two components A and X.


If we could make this dependency relation explicit there would be a way to constrain the A.B-X.Y relations on higher level.


This possibility would allow you to control the component-interdependencies - even if they are nested deeply in the component structure.

Technical Architecture

You may argue that component are not complex enough to control their use-relation by an explicit dependency. But imagine a software architecture which is based on layers and packages:

All packages and subpackages are somehow related to each other. Since the above diagram shows the technical architecture of our system the artifacts placed in the packages are templates (when generating) or simply code.

Just imagine you could explicitly define the dependencies between the packages. And now imagine that every template inter-dependency (one generic class using other generic classes) is checked immediately by the actifsource real-time validator.

Providing such a system your technical architecture would be documented soundly. And moreover actifsource would ensure that your code follows perfectly along the architecture.

Business Architecture

Another very interesting application of dependency relations is found when designing the business architecture. Coping with a complex business domain the relating business objects have to be structured to keep the overview.


Grouping domain-objects in packages and defining dependencies between the packages allows you to control use-relations between components from a higher level.

Providing such a system would allow you to group domain objects together for a perfect overview. Moreover actifsource would ensure that any relationship between components follows the well defined business architecture.
Conclusion

Modeling explicit dependencies allows you to control use-relations from a higher view. This view is what we usually call the architecture overview. actifsource could not only provide sound documentation of the architecture (technical and business) but also ensure at real-time that no artifact breaks the dependency rules.

The secret to this functionality lies in typed folders as Micha described in his blog Grouping Is Domain Specific Too. In one of my next blogs I try to workout the tricky details.

Friday, July 2, 2010

Writing C# using actifsource and eclipse

When people from the dotNet world hear about actifsource running on eclipse, they always ask us if it is possible to generate c# code for example. Today I want to show you how to create a c# application in eclipse using the emonic plugin.

First you have to install some prerequisites:

After you have installed them you need to setup emonic:


I only installed the "Eclipse Mono Integration", the debugger doesn't want to install at the moment and the other features are not required for this demo. When you have finished the installation, you first need to setup .NET preferences. Select the nant-binary-file:


and afterwards add a .NET-Framework Implementation. For example I used mono and chose the following settings:




Now create an actifsource projects for your meta model and the actifsource templates. I used the actifsource core model as meta model and wrote a simple data class template. The template is nothing special, but instead of writing java code, I wrote c# code:







Note that I have created a separate actifsource project for the actifsource template and generation specific resources. With our multi model concept, there is no need to modify your meta model project, just create an extension project. Since the actifsource project containing the template is a Java Project and mixing Java Code with C# Code is not recommended, switch to .NET-Perspective and create a separate .NET-Project.





I used "Mono 2.0" as target, you may use another target framework. This setting is important, since generics for example are not available in "Mono 1.0".

For generating the data classes I first need some actifsource classes. I just reused one of my last examples and put it into a newly created asrc folder in the dotNet project.



Now there are only a few steps to do, add a project dependency to the template-project and define the target folder.



The c# files will be generated immediately after pressing "ok". In emonic to make them compile, you have to right click on them and add each file to the build.



When doing this the first time, you need to set a build target:



Finally I added a small demo application which creates two instances of the class "Person", fills them and queries the data for writing it to the console.



To run the application one needs to right-click on the exe-file and select run .NET-Application.



That's it. If you don't want to add each file, you need to edit the build script and make use of wildcards. One drawback using wildcards is that the emonic builder doesn't seem to detect file changes for these resources.

Structural vs. Functional use-Relations

In my last blog I talked about the semantics of permanent use-relations. In the meantime Micha had the idea not to differ between permanent and temporary relations but between structural and functional relations.

The idea remain the same. Some use-relations are important for the overall component structure and some are not. Let's have a look at some examples.

First of all, let's have a look at the following structure. Component owns Command owns Argument. But the Argument itself has a use-Relations to type. Imagine a function declaration f(int a, float b) where the argument a has a type int and b has type float. The relation of a method argument to its type is simply functional an will never appear as a relation in an UML diagram.


The second example show a meta-model which declares that components may extend/subclass other components. This relationship is clearly structural because it's important from the component's view. Just imagine the specialization relation in a UML diagram.


Quite special on the above relation is the fact that it's never necessary to name the role of the extends relation more specifically. Extends remains extends no matter what is extended.

The next example shows a meta-model for a component which has a collaboration relation to other components. The difference to the above design is that the collaboration relation has a role which has further information (i.e. at least a name.) For that reason the role has to be designed as an own class on the meta level called Collaborator in the subsequent example.





Let me give a résumé: First of all we distinguish structural and functional use-relation. As a rule of thumb only structural use-relations matters for the overall component structure.

Taking a closer look at structural use-relations one notice that there are relations with and without roles. If no role is needed the design is simple as seen above. If A extends B the extends relation between A and B needs no further information.


But if we do need to distinguish roles for a structural use-relation this role has to be modeled in a separate class providing at least a name. In the example below component A uses B1 as xAxis and B2 as yAxis. Note that xAxis and yAxis are instances of Collaborator.



It's quite interesting that the class Collaborator of the meta-model is visualized as a role in the instance-model. Since the decision if a structural use-relation needs a role or not is taken by the designer, we have to distinguish 3 different modes for use-relations:

  • functional use-relation
  • structural use-relation without a role
  • structural use-relation with a role
If we specify the use-relation mode on the use relation it becomes possible to visualize components and their interdependencies soundly.

To conclude this blog let's think if this concept might work for a generic diagram editor which can visualize meta-model and model at the same time. For this reason let's have a look at the following simplified UML meta-model (which can be seen as meta-meta-model for our example).


  • The type-relation from Argument to Class is a functional use-relation
  • The extends relation from class to class is a structural use-relation without a role
  • The type-relation from Member to Class is a structural use-relation with a role (Member play the role)
At the moment it seems to me that our concept can be applied to all types of models. And this would be greatly simplify the development of the diagram editor.

Sunday, June 27, 2010

On the semantics of permanent use-relations

3 weeks ago I have written a blog with the Title Domain-Specific Presentation is Component Visualization. The idea was to provide a domain-specific component visualization which is aware of the different types of components and their relations.

Since then I've asked myself if there isn't any UML-way to visualize domain-specific components. I know how this sounds like: UML is generic by its very nature but not domain-specific. But let's give it a try anyway.

Before starting I have have to explain one thing first: actifsource distinguish only between two types of relations: the own-relation and the use-relation. The own-relation defines a strong coupling. If A owns B, B must not exist without A. The use-relation on the other hand doesn't imply any life-time semantics.

As I explained in my blog Software Architecture and Components, domain-specific components are defined implicitly around the own-relations of their meta-model.

Looking at the UML Component Diagram we find out that relations between components are realized as delegation connectors which connects a port to an element of the component.


And now the interesting question: does every use-relation between components lead to a port?

To answer this question, lets have a look at the meta-model of a UML class. A class can be seen as a component as well, composing

  • a name attribute
  • member variables
  • member functions
Lets have a closer look at member variables which can express to different relations:

  • own relation
  • use relation
Unfortunately, using 3rd GL's the semantic isn't always that clear. Programming in C++ I use pointers to imply use-relations and references or value-objects to imply own-relations whenever possible.

Before continuing the discussion, let's have a look at member functions. Functions consist of named and typed arguments. The types are modeled as a use-relation to other existing types.

So, what is the difference between the use-relation of member-variables to other types and the use-relation of typed arguments of a member function?

The use-relations of member-variables define the permanent relations between components while the use-relations of typed arguments of a function definition simply simply define a temporary relation.

Obviously we have to distinguish between two different of use-relations:

  • use-relations which define permanent relations between components
  • use-relations which define temporary relations between components
For a valuable component relation overview we are interested only in permanent relations between components which leads to the following definition:

Permanent use-relations shall be shown as ports in a component diagram.
In my next blog I try to explain how composite structures shall be visualized using the UML Composite Structure Diagram combined with the idea of ports for permanent use-relations.

Friday, June 25, 2010

Be careful when using access modifier

When working with frameworks provided by others, I sometimes get a little bit frustrated. Using the "normal" features provided by a framework in general is no problem. But when I have to access more information and go deeper into the code, I often find package-private and private modifiers protecting the code I like to reuse. Most often the problem is not at the method level, the method needed to call is public. In many cases even the constructor of the class is public, the problem begins with the arguments of the constructor or the method.

Oft one of the arguments is a class declared package-private or is public but has a package-private constructor. This forces me to search for a factory-method, which is often hidden deep by other such constructs. Meaning the code is strongly coupled to the framework.

Here is a small example:



I hate such code, especially if there is no special reason why the factory has a package-private constructor. Maybe there is an misunderstanding between information hiding and class design.

In cases where the code doesn't access any data from the framework there is no need to make the constructor package-private. It doesn't matter, if there is a second factory used by someone else. If you really need to have to use package-private, please make use of interfaces. This way it remains still possible to reuse the class by creating a new factory implementation:



I always wonder how the test code for such strong coupled code looks like. If you use interfaces unit testing is an easy task. You might use a mock framework like easymock. Using interfaces your class can be tested independent from the factory implementation and without creating an instance of the framework class.

For easier testing and better reusability, I suggest to use public-interfaces for a reasonable decoupling whenever possible.

Thursday, June 24, 2010

Abolish the Mutants! .. or do we need Setable and Mapable Interfaces?

No, this is not about a well-known film series. This is about software development! Forgive me for the lurid title :-)

Some weeks ago I was writing about an unexpected behaviour when you use Map.Entry objects in HashSets. The problem was emerging because the code assumed an object to be immutable that was not. More generally, we can ask: What elements can we put into a Set or use as the key of a Map?

In the javadoc to java.util.Map we find following warning:
Note: great care must be exercised if mutable objects are used as map
keys.  The behavior of a map is not specified if the value of an object is
changed in a manner that affects equals comparisons while the
object is a key in the map.
Unfortunately, we usually do not immediately see wheter an object can be changed in such a way or not and therefore are clueless whether we are allowed to use it as the key of a mapping. We need to check the implementation of the class and possible sub-classes. If we are dealing with an interface we would have to check all implementations which is an impossible task as new implementations might be added in the future.

The inventors of the java collection library could have diminished this problem by introducing an interface Setable resp. Mapable which mark whether a class is designed for being put into a set resp. for being the key in a map. Map and Set could then allow only objects implementing these interfaces. However, these interface names are rather cumbersome and as they choose not to do this, we need to find another way of dealing with this issue.

Universum java est omnis divisa in partes tres

If we look at the universe of all java objects we can group them into three distinct sets:
  1. Immutable objects
  2. Entities
  3. Neither 1 nor 2
Immutable objects are objects which never change their internal state. They are created fully-initialized, their hash code never changes and the equality relation formed by their equals method remains the same in their full livetime. They are perfect set members and hash keys!

Entities can change their internal state over time but they have a constant identity-giving property. Every java class which does not override equals and hashCode from java.lang.Object fullfills this requirement: It's identity is given by it's address in memory (which is only accessable to the jvm but nonetheless it is consistently defined). In other cases, the entity's identity is given by some id which might be a final long field or some special guid. The equals and hashCode methods must then be defined on this id only and not on any other fields. This makes entities suitable for sets and maps.

The remaining objects which are neither immutable nor entities can change their identity over time. I will therefore call them mutants for now. We should use them in sets and maps only if we can somehow guarantee that they will never mutate (change their identity) the entire time while they are used in the sets or maps!

Avoid Mutants!

Unfortunately, there are many mutants hiding in the java libraries:
  • java.util.List and all its implementations
  • java.util.Set and all its implementations
  • java.util.Map and all its implementations
  • java.util.BitSet
  • java.util.Date
  • ...
So, what can you do if you want to use a list of elements safely as key to a map, for example?

public class Registry {
  private Map<List<String>, String> map = new HashMap<List<String>, String>();

  public void put(List<String> strings, String value) {
    // oops, unsafe!
    map.put(strings, value);
  }
}

The list of strings is a potential mutant!


  public void put(List<String> strings, String value) {
    // oops, still unsafe!
    map.put(Collections.unmodifiableList(strings), value);
  }

Wrapping the mutant into an unmodifiable list will still not work as the caller of the method still has a reference to the list and can therefore modify it!


  public void put(List<String> strings, String value) {
    map.put(createImmutableCopy(strings), value);
  }

  private List<String> createImmutableCopy(List<String> strings) {
    return Collections.unmodifiableList(new ArrayList<String>(strings));
  }

We need to both copy the list and put it into a unmodifiable wrapper! Now we can guarantee that this list will never change again because there are no references to the wrapped ArrayList except by the Collections.UnmodifiableList wrapper which guards it from modifications. Note that we have this guarantee only because we know that the list is an direct result from our helper function. If we pass the list to some other method it will have to be treated as potential mutant there again...

As you see, mutants cause a lot of trouble and work to ensure they do not change their identity again. Therefore my demand is cleary: Do not write any mutant classes! By cleverly compositing immutable classes and entitiy classes you can always avoid the danger-bringing mutants.

Friday, June 18, 2010

Generating an internal DSL from an external DSL

We have learned that when we create sound explicit intension revealing APIs the number of WTFs per minute is reduced and the programmer's developer life is a bit easier.

Further we can introduce a steep learning curve and make less faults due to accidental missusage of the API.

Making internal DSLs formulated in our general purpose programming language is one way to aproach this but writing an internal DSL is not an easy task and violates the KISS principle.

But since you want to make the life of our developers a bit easier and even more fun it makes sense to create such an internal DSL where the developer can declaratively describe what is needed from the API.

actualValue = ... // this value should be detected by the framework
Detector detector = actualValueFramework.bind(new DetectorModule {
@Override
public void configure(DetectorExpressionBuilder detector) {
detector.detect()
.withEvent().change()
.andInterval().humanVisual()
.andPersistency().disc();
}
}).to(actualValue);

Listing 1

My prefered internal DSL style in Java is shown in listing 1 and is known as Method Chaining.

With code completion from the IDE this is very neat since the proposed possibilities from the completion express the further declaration steps. The developer can explore the API without any prior knowledge of its usage.

So this way I can achieve the goals mentioned above. Exploring is fun. Code completion while exploring makes it easy. Finding words from the problem domain makes it intension revealing.

But how can I make my own life easier and even more fun?

I have learned that the internal DSLs in Java can have a generic domain model, a meta-model. There is always a variation-point and its variations. Variations can be defined using polymorphism. A generic domain model must therefore give me the ability to express my API this way.

In the example above I have an Domain-Event that can be either Change, reach an Upper or Lower Limit. Then there is an interval that can be a millisecond, humanVisual oder oneMinute. The persistency can be disc or memory.

Event, interval and persistency are the variation-points where as change, upper, lower, millisecond, humanVisual, oneMinute, disc and memory are variations.

Using actifSource I created an external DSL that lets me express this variability in a generic domain model and write a code generator that generates ExpressionBuilders from the specific domain model.

Now it is no more hard to write such type of internal DSL where your specific domain model can be expressed in a declarative descriptive way.

How is it done using actifSource?

On the generic domain model class ExpressionBuilder is defined one property called operation and one called expression.

The property operation is from type Operation and expression is from type Class.

Operation is a NamedResource and here the operation name is used as a starting point of the expression. As in listing 1 the detector.detect() method. The Class contains a Class Instance having defined its own properties. Each property's Range is from an abstract class.

The Class is the variation-point where as the Class Instance is the variation.


public PersistencyExpression withPersistency(){
return new PersistencyExpression();
}


Listing 2

The generator reads each property and defines a with-Method as shown in Listing 2 for
i.e. the Persistency variation-point.

Further the generator generates an ExpressionBuilder by looking up all instances of the type defined as Range on the property and creates methods.


public class PersistencyExpression{
public AndProperties memory(){
persistency = new Memory();
return new AndProperties();
}
public AndProperties disc(){
persistency = new Disc();
return new AndProperties();
}
}

Listing 3

The end declaration of the internal DSL expression is also generated as shown in listing 4.


public Detector build(){
DetectorImpl build = new DetectorImpl();

if(persistency!=null){
build.setPersistency(persistency);
}
if(interval!=null){
build.setInterval(interval);
}
if(range!=null){
build.setRange(range);
}
if(event!=null){
build.setEvent(event);
}

return build;
}


Listing 4


public class ActualValueFrameworkImpl ...
public BindTo bind(final DetectorModule detectorModule) {
return new BindTo(){

@Override
public Disposable to(final ActualValue actualValue) {
DetectorExpressionBuilder detect = new DetectorExpressionBuilder();
detectorModule.configure(detect);
return detect.build();
}
};
}
...
}

Listing 5

Now the framework method looks like listing 5.

If this is a proper usage of modeling or not... it does make my life easier an I can generate several internal DSLs using this aproach and have nothing else to do as to declaratively define it and press generate.

Happy modeling, Nils

Plugin Dependency Management outside Eclipse

Today I searched for a solution to simplify using the actifsource ant task. Until now, all actifsource scopes (either an eclipse project or a bundle) have to be defined using a special actifsource ant-task, which takes a classpath. When making heavy usage of bundles, these classpaths can really become long and updating them by hand is not very convenient.

One idea I had to improve this is to define the plugin-directory directly and only point to the required bundles by their symbolic names. To do this we need to load the plugin outside of eclipse. The first place to start was to have a deeper look into equinox and see if there is a way to simply load all plugins and use them to load the required classes. At the moment we don't need to activate the bundles.

Sadly the equinox framework seems to strongly rely on some properties and requires the existence configuration files on the files ystem. Reusing the implementations like BaseAdaptor, Framework, HookRegistry and BaseStorage isn't really an option since they are strongly bound together. Some of the classes have only package private constructors, others have public constructors but taking one of the private classes as parameter. Time to search another solution. OSGi is an open standard and every implementation should be able to load the bundles.

The solution I found, was Apache Felix. It was said to be easy in setup and when I tried an example I was really glad to find out that this is true.

The following code allows me to load a bundle located in the plugin directory:

Felix felix = new Felix(Collections.emptyMap());
try {
felix.start();
} catch (BundleException e) {
e.printStackTrace();
return;
}
File bundleDir = new File("C:\\eclipseLocation\\plugins");
Bundle bundle = felix.getBundleContext().installBundle("locationId", new FileInputStream(new File(bundleDir, "bundle.jar")));


This is really short. Using this approach to load all bundles from the bundle-Directory would allow us to search for the bundles by their symbolic name and directly refer to them. No more classpath setup in every ant-file. Maybe we could also load some extension point definitions and and simplify the whole registration of generator tasks.