Janino, an embedded compiler for the Java programming language.

Architectural Overview

JANINO is a non-trivial piece of software, so I'll depict its data model in several steps in order to explain the various aspects.

(Fictitious) utmost simplified version of org.codehaus.janino.Compiler

This compiler reads .java files, compiles them, and creates .class files.

Java.CompilationUnit Java.PackageMemberTypeDeclaration[] Java.Annotation[] Java.TypeParameter[] Java.FieldDeclaration[] Java.MemberTypeDeclaration[] Java.ConstructorDeclarator[] Java.MethodDeclarator[] Java.Annotation[] ... ... Java.Annotation[] Java.TypeParameter[] Java.BlockStatements[] on command line .java files o.c.j.util.ClassFile ClassFile.AttributeInfo[] ClassFile.FieldInfo[] ClassFile.MethodInfo[] ClassFile.AttributeInfo[] ClassFile.AttributeInfo[] .class files in dest dir o.c.j.Parser parsed by into o.c.j.UnitCompiler compiled by into written to

As you know, a ".java" source file can contain multiple type declarations (exactly one of which must be PUBLIC), so that's why the Java Language Specification (JLS) calls a ".java" file a "compilation unit".

The org.codehaus.janino.Parser (abbreviated as "o.c.j.Parser" in the figure above) parses the compilation unit, understands the Java grammar, and creates a tree of objects that exactly reflects the contents of the compilation unit (except some irrelevant aspects like (non-JAVADOC) comments). The class names of the objects of the syntax tree are all " org.codehaus.janino.Java.*".

Like the Java.CompilationUnit structure is a one-to-one representation of the parsed compilation unit, the org.codehaus.janino.util.ClassFile structure on the bottom right of the figure is a one-to-one representation of a class file. (If you're not familiar with the Java class file format, read the respective section in the "Java Virtual Machine Specification".)

The UnitCompiler's job is to translate the compilation unit into class files. For example, a Java.MethodDeclarator is transformed into a ClassFile.MethodInfo, the Java.BlockStatements of the method declarator into a ClassFile.CodeAttribute of the ClassFile.MethodInfo object, and so forth.

After processing all input files this way, we have a set of class files that are ready to be loaded into a JVM.

Job done? No. This compiler would be very limited, for two reasons:

  1. It cannot "see" classes outside the compilation unit (not even java.lang.Object and java.lang.String). Not good.
  2. Java allows references between classes, and even circular ones! Because our compiler "compiles classes one after another", such references are not possible.

Effectively, this compiler is not of much use. It could only compile fields and methods that use primitive types (boolean, int, ...). So we must do better!

Enhanced compiler version that uses "resolved classes"

First we address the problem of arbitrary references within one compilation unit. The trick is, while compiling one type declaration, to look at the other type declarations in the same compilation unit (whether they are already compiled or not). But because the "other" type declaration is not yet compiled, its superclass name, implemented interfaces names, fields types, method return types, method parameter types asf. are not yet "resolved". E.g. for

    package my.package;

    import com.acme.Person;

    class Class1 {
        public void
        myMethod(String s, Person p, Car c) {
            // ...
        }
    }

    class Car { ... }
  

, the type names "String", "Person" and "Car" have not yet been resolved into " java.lang.String", "com.acme.Person" and "my.package.Car".

Therefore, a third representation of a class/interface must be introduced, which (for JANINO) is the IClass hierarchy:

Java.CompilationUnit on command line .java files o.c.j.IClass IAnnotation[] IField[] IConstructor[] IMethod[] ... ... ... IAnnotation[] IClass returnType IClass[] parameterTypes IClass[] thrownExceptions ... o.c.j.util.ClassFile .class files in dest dir o.c.j.Parser parsed by into o.c.j.UnitCompiler compiled by into uses written to

An IClass represents the "outer face" of a usable class or interface, which JANINO needs when that class or interface is "used".

Now where do we get the IClasses from? A complete implementation requires three different sources:

Java.CompilationUnit on command line .java files .java files on sourcepath o.c.j.IClass o.c.j.util.ClassFile .class files in dest dir o.c.j.ClassFileIClass .class files on classpath o.c.j.util.ClassFile o.c.j.UnitCompiler.resolve() o.c.j.Parser parsed by wrapped by into as a o.c.j.UnitCompiler compiled by into uses finds implements- written to o.c.j.ResourceFinderIClassLoader parsed into wrapped by as a
  1. To resove "Car" (declared in the same compilation unit), the UnitCompiler uses its "UnitCompiler.resolve(org.codehaus.janino.Java.TypeDeclaration)" method (bottom left), which wraps a parsed type declaration as an IClass.
  2. To resolve "java.lang.String" (found on the compilation classpath), the UnitCompiler uses an animal called the "org.codehaus.janino.ResourceFinderIClassLoader", which searches the classpath for a resource named " java/lang/String.class", loads it via "ClassFile(java.io.InputStream)" and wraps it as an IClass (bottom right).
  3. To resolve "com.acme.Person" (declared in a different compilation unit), the UnitCompiler searches and finds a resource "com/acme/Person.java" on the sourcepath, parses it (center) and uses "UnitCompiler.resolve(org.codehaus.janino.Java.TypeDeclaration)" to wrap the type declaration "Person" as an IClass.

And Bob's your uncle! That is everything that org.codehaus.janino.Compiler does.

Using loaded classes instead of parsing class files

Typically, to compile a set of compilation units, many other required classes have to be loaded and parsed via the compilation classpath. This costs a considerable amount of time and memory.

For "embedded" applications, i.e. when you want to compile and load classes in the same running JVM, it is much more efficient to use the loaded required classes, instead of parsing class files. Basically that is what the org.codehaus.janino.SimpleCompiler does:

Java.CompilationUnit or code snippet .java file o.c.j.IClass o.c.j.util.ClassFile java.lang.ClassLoader The running JVM's o.c.j.ReflectionIClass java.lang.Class o.c.j.UnitCompiler.resolve() o.c.j.Parser parsed by wrapped by into as a o.c.j.UnitCompiler compiled by into uses java.lang.ClassLoader.defineClass() implements- loaded via into o.c.j.ResourceFinderIClassLoader ClassLoader.loadClass() finds via wrapped by the as a

The org.codehaus.janino.ClassBodyEvaluator, org.codehaus.janino.ScriptEvaluator and the org.codehaus.janino.ExpressionEvaluator are merely variants of the SimpleCompiler that call, instead of Parser.parseAbstractCompilationUnit(), the Parser.parseClassBody(org.codehaus.janino.Java.AbstractClassDeclaration) method, resp. Parser.parseMethodBody(), resp. Parser.parseExpression().

Packages 
Package Description
org.codehaus.commons.compiler
This package declares interfaces for the implementation of an IExpressionEvaluator, an IScriptEvaluator, an IClassBodyEvaluator and an ISimpleCompiler.
org.codehaus.commons.compiler.io  
org.codehaus.commons.compiler.java8.java.util
A set of (rudimentary) proxies for Java-8+ classes that also compile for Java 6 and 7.
org.codehaus.commons.compiler.java8.java.util.function
A set of (rudimentary) proxies for Java-8+ classes that also compile for Java 6 and 7.
org.codehaus.commons.compiler.java8.java.util.stream
A set of (rudimentary) proxies for Java-8+ classes that also compile for Java 6 and 7.
org.codehaus.commons.compiler.java9.java.lang.module
A set of (rudimentary) proxies for Java-9+ classes that also compile for Java 6-8.
org.codehaus.commons.compiler.jdk
An implementation of the org.codehaus.commons.compiler API that uses the "JAVAC" Java compiler that is part of the "Java Development Kit" (JDK).
org.codehaus.commons.compiler.jdk.util  
org.codehaus.commons.compiler.lang  
org.codehaus.commons.compiler.samples
Sample applications for the Janino Java compiler.
org.codehaus.commons.compiler.util
Utility functionality for this project.
org.codehaus.commons.compiler.util.iterator  
org.codehaus.commons.compiler.util.reflect
Utility functionality related to java.util.reflect.
org.codehaus.commons.compiler.util.resource
Classes related to loading "resources" (ResourceFinder) and creating resources (ResourceCreator).
org.codehaus.commons.nullanalysis
Annotations for ECLIPSE's "null analysis" feature.
org.codehaus.janino
The core of the Janino Java compiler.
org.codehaus.janino.samples
Sample applications for the Janino Java compiler.
org.codehaus.janino.tools
Auxiliary command line tools related to JANINO.
org.codehaus.janino.util
Application-independent helper classes.
org.codehaus.janino.util.charstream
Application-independent helper classes.
org.codehaus.janino.util.signature
Application-independent helper classes.