Introduction

With IzPack 5, there has been a gig step done and great effort invested by several people to build the codebase and build system on scratch, based on picocontainer and dependency injection and Maven. The picocontainer framework helped us a lot to clean up the code by involving implicit instance creation all over the place, during compiling, unit testing and installing.

Nevertheless, the code is currently at an architectural obstacle we cannot overcome without refactoring it another time and undergoing some radical changes.

There are some obvious drawbacks in IzPack 5 (to be enhanced):

• The biggest one is missing modularity. The compiler and generated installers, although optimized in size, are still a monolithic pieces of software. We put all runtime dependencies in an installer to one classpath and cannot handle for example different versions of one and the same framework in several custom implementations a user might want to add. All contents of libraries provided by <jar> tags are merged to the same classpath like the built-in installer code and loaded by one and the same classloader all at once. There is no per-feature separation for them and all is loaded at once regardless of what is actually needed for a given scenario.
• No custom module (features, plugins) which can be added and loaded at once depending on the needs as one piece.
  For example you might want to add a feature to your installer deploying a WAR file to a running Tomcat container. Therefore you need to gather the Tomcat Manager parameters (host, port, credentials), translations, some Tomcat Manager client libraries (maybe also for different versions of Tomcat), a connection validator, a dedicated custom "listener" to do the actual deployment and so on. Currently, all these things, panels, validators, listeners and client libraries must be defined somewhere in the install.xml, UserInputSpec.xml, translation packs and so on in different tags and files. At the end, without good comments you can't recognize any longer which XML elements and libraries actually belong to your initial feature.
  Instead, there should be a way to add a jar containing your own custom implementation including its  dependencies, which can be easily added and automatically integrates to your installer, without the needs of defining additional <jar> and many more tags and resources explicitly.
• We don't use sufficiently modern technologies built-in to the JDK/JRE, like JavaFX/OpenJFX for panel layout, Java Preferences API for registering instead of focusing to native Windows registry methods, NIO etc. There is a lot of code still based on JDK 1.4 and previous Java versions. According to our needs we might also increase the JDK and JRE requirement to throw aboard the support of JDK versions which haven't received (security) updates for ages. Nevertheless, there should be again a long-term support for a defined JDK/JRE.

I'd like to reuse as much as possible of the work on the code, translations and documentation that has been done since the project has been started. I'd just like the house to be moved to another basement, including the the changes implying after doing so.

The core messages of this new IzPack generation for architects and implementers should be:

• Clean up the design of the good features.
• Focus as much as possible on the features the native JDK offers.
• Avoid JNI were it is possible. For OS-dependent operations use standard system commands.
• Use and migrate to the new codebase, probably a OSGI-based one.

Ideas of a new codebase

• A prerequisite to use JDK 1.8 at compile and runtime is a very probable option.
• Regarding modularity, OSGI will be probably a "must have". 
  We might use an embedded, auto-started OSGI container based on Apache Felix, even better in size is Eclipse Concierge to keep it small.
  We might also consider to support external OSGI containers which hold some dependencies on a given system and help to keep the installer jar size small.
  We won't wait for JDK 9 and Jigsaw, furthermore Jigsaw doesn't support versioned bundles. We can combine it later with Jigsaw, but this is not subject to a discussion now.
  OSGI services should provide and consume (to be enhanced later):
  • IzPack variables / dynamic variables
    (add new variables, modify and read values)
  • IzPack conditions
    (add new conditions, verify whether a condition is true)
  • Translations / language packs
  • ...
• Along with the above point there appears the question of whether to support pre-installed IzPack runtimes, regardless whether they would come as standalone libraries, as a OSGI container the installer might start and register to or maybe even in common application containers, like Jetty, Tomcat, etc. This is just a rough idea, because all native installers like MSI on Windows, Snapper, RPM or DEB for Linux profit from a comprehensive pre-installed runtime, scripting engines etc. processing the dedicated package, which then has just content, but no installation code. An IzPack runtime could be installed by the izpack-dist installer as an option. At the moment, it provides just tools for creating an installer, it might also contain a pack with a runtime required to run a dedicated installer.
  Nevertheless, there should be always the possibility to compile a standalone installer jar just requiring a pre-installed JRE.
• Should we use Spring?
  I'm currently in doubt with this. As a plus, Spring offers dependency injection (could replace picocontainer).
  But t's OSGI support is going to be discontinued. Further, it generates a big overhead, not really needed, if you have just a "Hello world" program installer without special features. For example, I've set up the simple example at https://projects.spring.io/spring-boot/, the compiled standalone demo-0.0.1-SNAPSHOT.jar is big, 6,2 MB in size. This is totally uncompetitive to the overhead a clean IzPack 5 installer generates. I wouldn't like to offer such a dinosaurus as compiled installer jar, although most real-world installers are usually much bigger in overhead just due to additional libraries, like JDBC drivers, added as <jar> tag.
  I see Spring still as a set of tools made for web applications and J2EE. I still don't have an idea how to design a reasonable rich client like a software installer with it (except of the useful OSGI support and dependency injection).
  With this state of knowledge I'd rather vote no, but who knows. 
• What is the best rich client framework for our purpose to make it easier to create custom panels? Should it still be plain old Swing, or JavaFX/OpenJFX 8, or even a well-supported 3rd-party one?
  Should the good old Swing-based UserInputPanel be replaced by a new panel type based on JavaFX/OpenJFX 8?
  Or should be use a light-weight browser and HTML, along with a light-weight, embedded servlet container, or support plain HTML and JavaScript? 
• What about picocontainer, will it work in a new architecture described above, especially with OSGI and containers?

Ideas of new functionality

• The main new feature I'd like to introduce and which will have affect to a series of other old-style features is again modularity.
  Ideally I'd like to see a whole set of whatever, custom panels and validators, "listeners", translations provided by a module (or feature, plugin or however we'll call it). Modules should contain a whole set of functionality for a special purpose, not just a single panel or action (listeners), but the whole set of panels, listeners, custom conditions and variables, validators, libraries etc. forming the implementation of that installer functionality. If you don't support the given functionality any longer, you should just remove a single definition in install.xml and all is done, not searching for <jar> tags, resources, conditions or variables which have been necessary to implement that functionality.
  A module should seamlessly plug-in to the installer core at compile and also at runtime (pre-installed features on the target machine). A feature module should be identified by a name and version. This is what the OSGI technology (bundles) should be used for. Modules should provide and consume services of different types.
  The installer core should have just a very minimal set of built-in modules. Instead the user should define in install.xml which features he/she wants to use.
  All what belongs to a give feature provided by a module should be defined in one place - at the module itself. There should no longer exist mixed definitions for specific enhancements into one monolithic install.xml.
  I'd like an approach of introducing modules in steps, not all stuff must be a module, but we can start for example with custom listeners, panels and validators, later we might also have module for functionality nearer to the Izpack core, like unpacking files, handling blocked files in Windows, etc.
  There should be a repository or page referring to third-party modules which can then be easily provided without fiddling with the internals of IzPack.
  Existing candidates for being moved to core functionality modules coming with IzPack are:
  • translations (language packs)
    The packslang.xml resource should come with a module containing all *PacksPanel implementations
    The customLangPack.xml resource should come with the modules providing code which uses them.
  • all internal panel types (GUI and console implementation)
    PacksPanel, TreePacksPanel etc. should probably be in one module come along with the translations currently provided by packslang.xml.
  • most internal conditions
  • internal dynamic variable types
  • AntActionInstallerListener (along with its uninstaller listener counterpart) including its dependencies (ant.jar, ...)
  • BSFInstallerListener (along with its uninstaller listener counterpart) including its dependencies (Rhino and other scripting languages and their transitive dependencies)
  • ConfigurationInstallerListener
  • installation mode (GUI, console, automated, mixed)
  • new panel and validators to choose a JRE (and assign a variable)
  • new JavaFX-based panels

• I'd introduce some kind of installation lifecycle (like a lifecycle in Maven), which should make the order of what is executed easier. Also modules should be better usable this way.
  The Maven model can be a good model also for IzPack - the installer executes lifecycle phases and modules can be bound to that phases and configured in their own way by configuration tags each module offers to be parsed and processed.
  The installation lifecycle should be definable for each installer at compilation time, there might be a default lifecycle covering most use cases. We might also consider modules to be able to add lifecycle phases dynamically.
  Like in Maven, lifecycle phases should get a name, like initialize, interaction, preinstall, install, shortcuts, postinstall etc.
  It should be possible to execute one and the same module at different lifecycle phases with a different configuration, exactly, just like Maven does, we must not reinvent the wheel here.
  An installation based on Izpack 5 is InstallPanel-centric, in future an installation should be lifecycle-centric, with loosely coupled actions (former listeners) and panel activations in a defined order. 
• Along with the idea of an installation lifecycle I'd like to decouple the functionality of panels and implicit actions. At the moment in IzPack 5, the central activator of actions is the InstallPanel (before and after unpacking, and that's all), except of some actions that can be optionally done also by the ProcessPanel and the ShortcutPanel implementations. This is not much flexible. I'd like to loosely bind activation of panels and actions to a flexible lifecycle mentioned above. This would allow to insert actions easily and repeatedly between activations of different panels, for example an Ant action to start an external application server before the live deployment credentials are asked in a user input panel, to be able to verify their validity when eaving the panel. The importance of the InstallPanel should be decreased to showing the progress of an ongoing unpack action. The listeners pattern should be transformed to installation actions bound to every possible phase of the new lifecycle, not just before and after unpacking files like now currently initiated by InstallPanel. I would still keep to special lifecycle phases "before unpacking" and "after unpacking", however we will call them.
  Furthermore, it should be possible to unpack files for dedicated packs separately, embedded to several other possible panel activations and or pack-specific actions, not all at once without any other interaction. At the moment, you can just bind "beforePack" and "afterPack" phase listeners, nut just if the listener implementation implements these actions. In future, binding actions to packs should be done by stepping into special lifecycle phases bound to packs generally and again being able to open additional panels or just calling Ant or BSF scripts, whatever.
• The Java Setup API should be used to save or load per-user or per-system information generated by the installer locally. This includes saving to the registry (Windows) and/or several configuration files (UNIX, MacOS).
  This feature should completely replace the handling of .installationinformation, which I would remove completely. Information about installed packs and user choices should be generated by the IzPack core functionality and also by modules and saved in a transparent way.
  This kind of information should be read when starting the installer and saved when the installation succeeds automatically. Each module can provide and request local system information about a special OSGI service its registers to.
• I'd like to keep the principles of <dynamicvariables> and <conditions>. They are well proven and suitable for our needs.
  There should be a built-in basic set of variable and condition types. Variable and condition types should be enhanceable by modules (custom conditions and variables).
  I'd would make <variables> and <dynamicvariables> as one new common tag - <variables>, using the syntax of the current <dynamicvariables> tag. Dynamic variables do already cover the feature of <variables>, there is no need to have their tags divided.
  The access to variables and conditions should be made available to all modules using OSGI services.
• There is one thing I wanted to remove from IzPack already a long time - the need of having named resources for a special purpose, like "packslang.xml_<iso3>" for package translations, "AntActionSpec.xml "for setting up the AntActionInstallerListener and so on. These resources should be provided as part of a module, and their configuration should be done at a very separate place, even if the internal format of these descriptors will mostly stay the same.
• I'd remove the <jar> tag from install.xml - external libraries should be provided in a module, along with their code or descriptors that make usage of these libraries.
• I'd remove the Registry(Un)InstallerListener including the CoiosHelper DLLs and would replace them by tools provided by the configuration API (which calls Windows system commands instead of a JNI implementation we have to maintain for all Windows versions) along with using the Java Setup API like described above. Writing independently to the Windows registry should not be offered out of the box in IzPack. Instead, the information to be written to a local system should be collected from all modules and written in a transparent way independently of the OS.
• I'd like to support just one installer pack compression format, there is no need to support pack2000 or bzip from my point of view. I'd recommend to get rid of these options.
• Nice to have: Compiler support for generating diff updates, installers just coming with the changed files against the preinstalled files (and removing obsolete files), minimized in size.
• Nice to have: Making pack content more transparent from outside, for developers to analyze the contents of an installer (unserialized plain files in the jar or a viewer, TC plugin etc. to show and extract contents without installing them). An alternative could be an installer mode just extracting pack files (without variable sibstitution) to a given location and not calling other installation actions.
• In all XML descriptors, wherever a boolean value is expected, accept just "true" or "false", no "yes" or "no" any longer. Everyone should be able to manage this. And allows us to remove superfluous string comparisons from our code.