Config management and installation

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Config management and installation

Jeremy Boynes
A couple of questions keep coming up about Geronimo's configuration
management system; things like:

* How do I change things during installation?
* How do I change things when the server is not running?
* Are we going to give the configuration plans to users?
* WTF is all this stuff anyway?

I was going to include background here but thought it was easier to
place it directly on the wiki. It can be found at:

http://wiki.apache.org/geronimo/Architecture/ConfigurationManagement

With that in mind, here we go ...

During installation we are really doing three things:
1) Installing the kernel and configuration management system
2) Installing the configuration bundles we want this server to run
3) Performing local-server configuration overrides

The first is easy, just install the jars and any of their dependencies
in the appropriate place. The user will need to specify where that is,
and will need to specify the configuration management system being used
(e.g. what are the default Repository, ConfigurationManager and
ConfigStore implementations).

This gives us a minimal environment where we can install other
configuration bundles and which can run *inside the installer.* On to
phase two.

--- Phase Two

Which configuration bundles to install will depend on just what the user
wants this server to do: is it full J2EE or just the web teir? do they
want the deployment system configured? is this just an app client
environment? Based on the responses, the installer can build a list of
configuration bundles to install, and then use the environment built in
phase one to to install them.

This leaves us with a server environment built to the user's
specification with standard bundles installed. That may be enough, but
if there are local overrides needed, on to phase three.

--- Phase Three

Once the bundles have been installed, the user may still have local
overrides. For example, we may have installed a generic HTTP connector
listening on port 8080 and they may want to change that.

The installer has access to the Configuration instances it installed and
to the persistent properties of the GBeans they contain. Values
specified in the installer can be used to override those property values
simply by setting the GBean attributes. The GBeans do not need to be
started to do that so there is no risk of conflict with other things
running on the system.

When the installer shuts down its kernel the GBean state will be
persisted, ready to be restored the first time the user starts the server.

---

There is no requirement here to give the actual source plans to users.
They don't form a very friendly UI :-) and instead we should be using
steps in the installer for the local configuration. Is there an easy way
to make that modular (e.g. by including helper code in the bundle itself)?

Having said that, the plans do form the ultimate definition of what the
bundle will contain and I would include them in the distro as
documentation and so that users don't have to get them from a source
bundle. We did that with Gluecode's distro.

---

OK, I hope that makes some kind of sense. If it sounds vaguely familiar,
it should do - it is very similay to how the assembly module pulls
things together. I'd almost be tempted to describe installation as
'assemble-on-site'.

--
Jeremy

Reply | Threaded
Open this post in threaded view
|

Re: Config management and installation

caboteria
On Sun, May 15, 2005 at 01:48:54PM -0700, Jeremy Boynes wrote:
> I was going to include background here but thought it was easier to
> place it directly on the wiki. It can be found at:
>
> http://wiki.apache.org/geronimo/Architecture/ConfigurationManagement

Thanks, Jeremy, it's cool to get some insight into the "whys and
wherefores" that the code can't always convey.  I've got a couple of
questions about the wiki page, probably based on me failing to make a
required intuitive leap.  You make the point that Geronimo's
architecture is designed to support remote management, but I didn't
get a good sense of why or how the old (i.e. uninverted) architecture
fails to support remote management, especially given that most telecom
OSS/BSS systems are built on the old architecture, yet they seem to
allow for remote large-scale network node management.

I guess I don't understand how configuring applications is more or
less amenable to large-scale network management than configuring
servers (especially as a big NMS would call Geronimo itself an
"application").  Later in the page it seems like you're trying to
point out that Geronimo has more of a "pull" model in that it can pull
components from a trusted external source.  That would be cool in that
it would allow a fleet of Geronimo servers to do on their own a big
part of what a traditional NMS would do for other j2ee servers.

It would really help me (and probably others) "get it" if you could
provide an example of how the traditional architecture falls down and
the application-centric architecture succeeds.

Thanks,
Toby
Reply | Threaded
Open this post in threaded view
|

Re: Config management and installation

ammulder
In reply to this post by Jeremy Boynes
        I'll resond in a sec to the Wiki and the thread.  To summarize:  
I don't disagree with the goals, but I don't think we're offering
something that reaches them either -- and I think we've sacrificed a lot
of usability to get there.  I think we'd do well to put together a more
comprehensive list of our goals and then figure out the best way to work
towards them.  That said...

        I read the Wiki page.  I'm not sure I follow the stated principle
of "configuring applications not configuring servers".  For example, let's
say you want to deploy "web application v. 1.2.3 using Jetty v. 2.3.4
listening on HTTPS on port 8443".  It sounds like you're saying you could
deploy the web application with the note that it should require the stated
Jetty environment as a precondition.  I can see that would be nice.  No
need to check your 1000 servers to make sure that environment is present
-- simply shoot off your application bundle 1000 times and on each box it
will deploy if the environment is there and not if it isn't, and maybe you
get back a report "912 deployments successful, 88 failed", or maybe the
whole things rolls back (ha ha), whatever.

        But we don't offer that.  If some of the boxes have Tomcat, some
have Jetty on 8080 only, some have Jetty with SSL on 443, and some have
Jetty with SSL on 8443, the app will cheerfully deploy on the last three
groups but not the first one.  Further, if some of the boxes have the
wrong version of Jetty, but it happens to have been deployed with a
similar plan in the same order (thus getting the same config name and ID),
I think we'll cheerfully use that too.  So I don't think we offer any of
the advantages touted on the Wiki page.

        Well, I have to take that back to some degree.  You can install
1000 blank servers -- with no configurations except "System" and a
deployer (though without J2EE features how can you have an application
deployer?).  Then your deployment operation can deploy like 10
simultaneous configurations, including J2EE Features, Jetty, ActiveMQ,
Database pools, Applications, etc.  So you can deploy a perfectly
identical configuration of the whole stack in one shot to tons of servers.  
But I think you have to do it in multiple steps (J2EE features then
deployer then services then apps or whatever), so you probably lose any
atomicness, and it would still be easier to get it working on one box, tar
that up, sftp it to 999 boxes, and untar it there and then start it (or
use a common DB as the config store to begin with, which we also haven't
addressed AFAIK).  Perhaps you could replace all the guts of a running
server (remove apps, remove services, remove deployer, remove J2EE
features, then reverse the process with a new version).  But if the 1000
servers start in different states to begin with, that probably won't work
either, unless we add a "remove everything" command.  I'l not exactly sure
what need we're addressing, and if we could clarify that, it would
probably make the necessary features jump out a bit more.

(Do we yet have a deployer that lets you copy a "bundle" from one server
to another?)

-----------------

        As far as the thread quoted below goes, I'm in 100% agreement with
the facts.  If you want to change your ports, you must use the install
routine, configure the ports using the installer UI, and let it deploy the
customized configurations when first building your custom server
environment.

        However, I consider this to be bad not good.  It's a nice
convenience feature to be able to customize ports during installation.  
It's pretty bad that customizing the installation is the *only* way to
change ports.  It's dumb that our offline configuration and deployment
features are nonexistant.  I wince every time I say "well, uh, if you want
to change ports, uh, you can reinstall the product and be sure to use the
installer not the zip".  People say "that's so stupid!  I can't believe
it's been a year and you still can't edit config settings!"  I say "no,
really, it's a 'feature'...  Plus I'm sure we'll have a tool some day.  
Really, uh, the installer is pretty cool."

        I suppose I could tell them to edit the plan and redeploy the
"J2EE Server" configuration.  I don't like that option very much either,
especially if you can't do it without starting the server (or starting
enough of it to access the deployer).  Though I guess the installer does
that in order to deploy your custom version anyway (ick).  BTW, the
installer can't replace Jetty with Tomcat.  It can only add Tomcat.  It
can't ensure that you don't assign the same port to both.  It can't
disable or enable SSL or AJP.  If we give Jetty and Tomcat identical Names
then you could pick one of the two and the rest of the dependencies would
still be OK, but the installer can't enforce that either.  Sure, we can
probably create a more robust installer, that does more automatic editing
of plans, adds various validations, etc.  But nothing like that exists
today.

---------------

        I don't disagree with the principle of repeatable deployments and
the assurance that things run as intended.  I just don't feel that our
current implementation actually offers that, and it also doesn't offer the
ease of use features that would make it attractive to less "enterprise"
(less "1000 servers") customers.  It doesn't seem to live up to the needs
of either group.

        I'd like to start from the principles we're targeting and think
about what's really the best way to achieve them.  I think we can come up
with something better than we have now.

        For example, one possiblity:  it might not be a bad start to work
with the existing infrastructure but save to XML instead of Serilization
-- or save to both but rescan from XML if the XML timestamp is more recent
than the Serialization timestamp, or if there were any errors during
deserialization.  Even with no serialization you get nice configuration
bundles you can copy from server to server, and a user can edit the config
if it's wrong.

        Still, this is looking at trees not forest.  Maybe we should just
have a thread to talk about who we're targeting and what features we need
to offer to satisfy them, and then we can talk about what's the most
effective implementation to deliver the most value in our 1.0 release.  
Maybe we'll discover that we're on the path already and I just don't
realize it.  Maybe we'll discover there's a better way to reach the goals
you're after.  But I don't feel like we've really had that conversation.

Aaron

P.S. It's my opinion that we need to address this before a 1.0 release,
even once TCK testing is totally complete.  Most of it doesn't have any
bearing on compliance, so long as whatever the proposed solution is
doesn't actually break the plans that pass the tests.

On Sun, 15 May 2005, Jeremy Boynes wrote:

> A couple of questions keep coming up about Geronimo's configuration
> management system; things like:
>
> * How do I change things during installation?
> * How do I change things when the server is not running?
> * Are we going to give the configuration plans to users?
> * WTF is all this stuff anyway?
>
> I was going to include background here but thought it was easier to
> place it directly on the wiki. It can be found at:
>
> http://wiki.apache.org/geronimo/Architecture/ConfigurationManagement
>
> With that in mind, here we go ...
>
> During installation we are really doing three things:
> 1) Installing the kernel and configuration management system
> 2) Installing the configuration bundles we want this server to run
> 3) Performing local-server configuration overrides
>
> The first is easy, just install the jars and any of their dependencies
> in the appropriate place. The user will need to specify where that is,
> and will need to specify the configuration management system being used
> (e.g. what are the default Repository, ConfigurationManager and
> ConfigStore implementations).
>
> This gives us a minimal environment where we can install other
> configuration bundles and which can run *inside the installer.* On to
> phase two.
>
> --- Phase Two
>
> Which configuration bundles to install will depend on just what the user
> wants this server to do: is it full J2EE or just the web teir? do they
> want the deployment system configured? is this just an app client
> environment? Based on the responses, the installer can build a list of
> configuration bundles to install, and then use the environment built in
> phase one to to install them.
>
> This leaves us with a server environment built to the user's
> specification with standard bundles installed. That may be enough, but
> if there are local overrides needed, on to phase three.
>
> --- Phase Three
>
> Once the bundles have been installed, the user may still have local
> overrides. For example, we may have installed a generic HTTP connector
> listening on port 8080 and they may want to change that.
>
> The installer has access to the Configuration instances it installed and
> to the persistent properties of the GBeans they contain. Values
> specified in the installer can be used to override those property values
> simply by setting the GBean attributes. The GBeans do not need to be
> started to do that so there is no risk of conflict with other things
> running on the system.
>
> When the installer shuts down its kernel the GBean state will be
> persisted, ready to be restored the first time the user starts the server.
>
> ---
>
> There is no requirement here to give the actual source plans to users.
> They don't form a very friendly UI :-) and instead we should be using
> steps in the installer for the local configuration. Is there an easy way
> to make that modular (e.g. by including helper code in the bundle itself)?
>
> Having said that, the plans do form the ultimate definition of what the
> bundle will contain and I would include them in the distro as
> documentation and so that users don't have to get them from a source
> bundle. We did that with Gluecode's distro.
>
> ---
>
> OK, I hope that makes some kind of sense. If it sounds vaguely familiar,
> it should do - it is very similay to how the assembly module pulls
> things together. I'd almost be tempted to describe installation as
> 'assemble-on-site'.
>
> --
> Jeremy
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Config management and installation

Jeremy Boynes
In reply to this post by caboteria
toby cabot wrote:

> On Sun, May 15, 2005 at 01:48:54PM -0700, Jeremy Boynes wrote:
>
>>I was going to include background here but thought it was easier to
>>place it directly on the wiki. It can be found at:
>>
>>http://wiki.apache.org/geronimo/Architecture/ConfigurationManagement
>
>
> Thanks, Jeremy, it's cool to get some insight into the "whys and
> wherefores" that the code can't always convey.  I've got a couple of
> questions about the wiki page, probably based on me failing to make a
> required intuitive leap.  You make the point that Geronimo's
> architecture is designed to support remote management, but I didn't
> get a good sense of why or how the old (i.e. uninverted) architecture
> fails to support remote management, especially given that most telecom
> OSS/BSS systems are built on the old architecture, yet they seem to
> allow for remote large-scale network node management.
>
> I guess I don't understand how configuring applications is more or
> less amenable to large-scale network management than configuring
> servers (especially as a big NMS would call Geronimo itself an
> "application").  Later in the page it seems like you're trying to
> point out that Geronimo has more of a "pull" model in that it can pull
> components from a trusted external source.  That would be cool in that
> it would allow a fleet of Geronimo servers to do on their own a big
> part of what a traditional NMS would do for other j2ee servers.
>
> It would really help me (and probably others) "get it" if you could
> provide an example of how the traditional architecture falls down and
> the application-centric architecture succeeds.
>

I'll admit it's been a while, but the basic principal of an NMS as I
remember it is that the network elements themselves are dumb - they
basically do nothing by themselves, the configuration of each one is
controlled by the NMS. The NOC has a picture of how the global network
is operating and, using the NMS, can reconfigure the elements as needed
to maintain the quality of service.

This is the architecture we are trying to recapture in Geronimo.

In a massive deployment, each server basically comprises of only a
pristine kernel and a local configuration manager (analogous to element
firmware). When it comes online, it is recognized by the global
application manager and dynamically configured to play the correct role
in the global application environment (just like the NMS configures the
element). As workload conditions change, the central manager can
reconfigure each server as needed.

This is different to the traditional application server, think Jetty as
an example (I hope Greg does not mind me picking on him). There each
server maintains its own configuration and the administration model is
one of going to the server (physically, via ssh, JMX remoting or a web
console) and configuring it. The setup of each server is defined by its
local configuration not by the external controller.

This works well for single servers or small clusters, but does not scale
to massive deployments. However, the Geronimo model can be scaled down
simply by having a very simple manager running inside a standalone
server. For example, the current setup uses the FileConfigurationList to
automatically configure the kernel at startup so that it is runs the
last known running set of configurations.


A couple of final points. Yes, an NMS would call Geronimo an
application; but also, the Geronimo architecture itself does not
distinguish between configuration bundles for "system" services (e.g.
the J2EE stack) and those for "applications" (e.g. a config generated by
webapp deployment). The ultimate vision is that the kernel itself
becomes ubiquitous and it is the configuration bundles that get moved
around by some NMS.

Secondly, you can get a long way there by layering this on top of an
existing architecture. For example, using web services or JMX you can do
a lot of reconfiguration of running servers. However, we tried to build
this in at the very core.

Thirdly, there are commercial systems out there that work a little bit
this way. For example, (at least this is how I understand it), BEA
WebLogic has this central management model at a cluster level where from
a central console you can reconfigure any member server; IBM also has
this in WebSphere ND and goes further with XD. In both, AIUI, you
interact with the controller and not the individual servers.

Fourthly, the "pull" stuff is really intended for small to medium size
deployments (where there isn't the full NMS). The idea is that resources
can be staged to a trusted server in the production environment and that
the runtime servers will pull from there. It also helps in a development
environment where there are a load of unmanaged development servers;
developers can install the app the need to integrate/test with and not
have to worry about where all the dependencies will come from.

I hope that helps clarify things. We definitely need to do more of this
and rely less on the raw code.

--
Jeremy