[POLL] API and Implementation serialization compatibility and upgrading

classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[POLL] API and Implementation serialization compatibility and upgrading

David Blevins
I really want to get some feedback from the developers, users, lurkers, other projects and everyone on this subject.  It shouldn't be a taboo to talk about or considered a "can of worms" or a "hot button."  It affects the project at pretty fundamental level, so I think it would be good if we did our best to get feedback.

The problem:

  Upgrading apps between versions of Geronimo and it's integrated projects is not currently possible without a redeploy.

The facts:
  - Our deploy system and storage of deployed apps is serialization based.
  - At deploy time we build and Jetty web containers, Tomcat web containers, ActiveMQ queues/topics, OpenEJB ejb containers, Axis webservices, TranQL connectors and more
  - We serialize those objects which include not just what they consider to be their public APIs but also the private implementations of those APIs.

The tricky part:
  - We, qualified in alphabetical order, ActiveMQ, Axis, Geronimo, Jetty, OpenEJB, Tomcat, TranQL, et. al. cannot change our public API *or* private Implementation of our APIs and still deserialize objects from a previous version.
  - Unless we (the projecst above) agree not to bug fix, optimize, refactor, repackage, or reorganize our code in a way that would break deserialization of objects created with a previous versions of our code, it will not be possible to upgrade applications leveraging these projects without a redeploy.


PROJECT POLL:
  [ ]  We will do our best to ensure the implementations of our APIs are serialization compatible to future versions of our code.
  [ ]  We do our best to ensure our public APIs, but use of our implementations in such a way is not supported by us.

USER POLL:
  [ ]  I do not mind redeploying my applications on each upgrade of Geronimo et. al.
  [ ]  I do not want to start from scratch on each upgrade of Geronimo et. al.



-David


Reply | Threaded
Open this post in threaded view
|

Re: [POLL] API and Implementation serialization compatibility and upgrading

David Jencks-2

On May 14, 2005, at 7:28 AM, David Blevins wrote:

> I really want to get some feedback from the developers, users,
> lurkers, other projects and everyone on this subject.  It shouldn't be
> a taboo to talk about or considered a "can of worms" or a "hot
> button."  It affects the project at pretty fundamental level, so I
> think it would be good if we did our best to get feedback.
>
> The problem:
>
>   Upgrading apps between versions of Geronimo and it's integrated
> projects is not currently possible without a redeploy.
>
> The facts:
>   - Our deploy system and storage of deployed apps is serialization
> based.
>   - At deploy time we build and Jetty web containers, Tomcat web
> containers, ActiveMQ queues/topics, OpenEJB ejb containers, Axis
> webservices, TranQL connectors and more
>   - We serialize those objects which include not just what they
> consider to be their public APIs but also the private implementations
> of those APIs.

I think this is wrong.  IIUC we serialize gbean attribute values and
reference patterns.  We also serialize a few classes from the kernel to
hold all that stuff.  Depending on which module you are talking about,
the gbean attributes could be entirely plain java types (transaction),
plain java + a few "data only" classes (connector, jetty), large
numbers of implementation classes together with some java (openejb), or
the entire runtime object serialized (axis integration).

So, there are 2 dimensions of incompatibility: GBeanInfo definitions
and the classes of GBean attributes.  With care we might be able to
make gbean infos upward compatible by only adding attributes and
references and supplying in code default values when the new ones are
missing.  Data only classes are perhaps not all that likely to change
incompatibly if they are designed well enough to begin with.  The big
problem comes when we are using large parts of the implementation as
attributes.  It  might be worth considering if it would be practical to
make more of the attributes data-like.

>
> The tricky part:
>   - We, qualified in alphabetical order, ActiveMQ, Axis, Geronimo,
> Jetty, OpenEJB, Tomcat, TranQL, et. al. cannot change our public API
> *or* private Implementation of our APIs and still deserialize objects
> from a previous version.

I think what we can't change is the GBeanInfo definitions and the
classes used as attributes.

thanks
david jencks

>   - Unless we (the projecst above) agree not to bug fix, optimize,
> refactor, repackage, or reorganize our code in a way that would break
> deserialization of objects created with a previous versions of our
> code, it will not be possible to upgrade applications leveraging these
> projects without a redeploy.
>
>
> PROJECT POLL:
>   [ ]  We will do our best to ensure the implementations of our APIs
> are serialization compatible to future versions of our code.
>   [ ]  We do our best to ensure our public APIs, but use of our
> implementations in such a way is not supported by us.
>
> USER POLL:
>   [ ]  I do not mind redeploying my applications on each upgrade of
> Geronimo et. al.
>   [ ]  I do not want to start from scratch on each upgrade of Geronimo
> et. al.
>
>
>
> -David
>
>

Reply | Threaded
Open this post in threaded view
|

Re: [POLL] API and Implementation serialization compatibility and upgrading

ammulder
In reply to this post by David Blevins
        I'd like some more information before answering this poll.  
Namely, what exactly is serialized?  I guess I was under the impression
that "configuration information" was serialized.  You make it sound like,
for example, ActiveMQ itself is serialized.  Shouldn't we be able to
serialize a bunch of settings (port=61616, active transport 1 =
vm://localhost, active transport 2 = tcp://localhost:61616, ...) and apply
those to version N+1 of ActiveMQ, so long as it didn't remove settings we
use or add new required ones without defaults that we don't use?  Are we
just not doing that?  I guess I need to take a closer look.

        Also, wearing my end user hat, I'm not as concerned about this
issue as stated -- Every time I upgrade, for example, JBoss, I install it
into a new directory, reapply any configuration changes, and redeploy my
apps.  It might be different if the product offered an "upgrade in place"
install option.  Still, I upgrade pretty rarely.  So an easy upgrade path
is probably for me, at best, a "nice to have".

        I'm more concerned about the inability to conveniently edit
serialized configuration information, which I hope to take up on the other
thread that Jeremy was going to start.  That is a lot more of a common
issue for me -- more of a "must have".

Aaron

On Sat, 14 May 2005, David Blevins wrote:

> I really want to get some feedback from the developers, users, lurkers,
> other projects and everyone on this subject.  It shouldn't be a taboo to
> talk about or considered a "can of worms" or a "hot button."  It affects
> the project at pretty fundamental level, so I think it would be good if
> we did our best to get feedback.
>
> The problem:
>
>   Upgrading apps between versions of Geronimo and it's integrated
>   projects is not currently possible without a redeploy.
>
> The facts:
>   - Our deploy system and storage of deployed apps is serialization based.
>   - At deploy time we build and Jetty web containers, Tomcat web containers, ActiveMQ queues/topics, OpenEJB ejb containers, Axis webservices, TranQL connectors and more
>   - We serialize those objects which include not just what they consider to be their public APIs but also the private implementations of those APIs.
>
> The tricky part:
>   - We, qualified in alphabetical order, ActiveMQ, Axis, Geronimo, Jetty, OpenEJB, Tomcat, TranQL, et. al. cannot change our public API *or* private Implementation of our APIs and still deserialize objects from a previous version.
>   - Unless we (the projecst above) agree not to bug fix, optimize, refactor, repackage, or reorganize our code in a way that would break deserialization of objects created with a previous versions of our code, it will not be possible to upgrade applications leveraging these projects without a redeploy.
>
>
> PROJECT POLL:
>   [ ]  We will do our best to ensure the implementations of our APIs are serialization compatible to future versions of our code.
>   [ ]  We do our best to ensure our public APIs, but use of our implementations in such a way is not supported by us.
>
> USER POLL:
>   [ ]  I do not mind redeploying my applications on each upgrade of Geronimo et. al.
>   [ ]  I do not want to start from scratch on each upgrade of Geronimo et. al.
>
>
>
> -David
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [POLL] API and Implementation serialization compatibility and upgrading

Phil Steitz-2
Aaron Mulder wrote:
<snip/>

>
> Also, wearing my end user hat, I'm not as concerned about this
> issue as stated -- Every time I upgrade, for example, JBoss, I install it
> into a new directory, reapply any configuration changes, and redeploy my
> apps.  It might be different if the product offered an "upgrade in place"
> install option.  Still, I upgrade pretty rarely.  So an easy upgrade path
> is probably for me, at best, a "nice to have".
>
> I'm more concerned about the inability to conveniently edit
> serialized configuration information, which I hope to take up on the other
> thread that Jeremy was going to start.  That is a lot more of a common
> issue for me -- more of a "must have".
>
> Aaron

As a user, I pretty much agree with Aaron (both points), with the
following additional comment on the first one:

Major version upgrades are relatively rare events, but applying patches
/ service packs / whatever they may be called is more common and
sometimes needs to be done quickly with minimal disruption.  Having to
redeploy all apps on a server to which a patch is applied is not ideal.
Having strange things happen after patches are applied is even less fun ;-)

So I guess it all depends on what exactly we mean by "upgrade"...

Phil

Reply | Threaded
Open this post in threaded view
|

Re: [POLL] API and Implementation serialization compatibility and upgrading

Davanum Srinivas
In reply to this post by David Jencks-2
Just FYI, "entire runtime object serialized (axis integration)" was a
choice...usually Axis uses a deployment descriptor (even jonas uses
these xml based wsdd descriptors)....but the decision to use
serialized runtime objects was conscious one in Geronimo.

Yes, need more information...(or more time to think i guess). But i
guess we can force people to re-deploy when they upgrade...if that is
the only choice.

-- dims

On 5/14/05, David Jencks <[hidden email]> wrote:

>
> On May 14, 2005, at 7:28 AM, David Blevins wrote:
>
> > I really want to get some feedback from the developers, users,
> > lurkers, other projects and everyone on this subject.  It shouldn't be
> > a taboo to talk about or considered a "can of worms" or a "hot
> > button."  It affects the project at pretty fundamental level, so I
> > think it would be good if we did our best to get feedback.
> >
> > The problem:
> >
> >   Upgrading apps between versions of Geronimo and it's integrated
> > projects is not currently possible without a redeploy.
> >
> > The facts:
> >   - Our deploy system and storage of deployed apps is serialization
> > based.
> >   - At deploy time we build and Jetty web containers, Tomcat web
> > containers, ActiveMQ queues/topics, OpenEJB ejb containers, Axis
> > webservices, TranQL connectors and more
> >   - We serialize those objects which include not just what they
> > consider to be their public APIs but also the private implementations
> > of those APIs.
>
> I think this is wrong.  IIUC we serialize gbean attribute values and
> reference patterns.  We also serialize a few classes from the kernel to
> hold all that stuff.  Depending on which module you are talking about,
> the gbean attributes could be entirely plain java types (transaction),
> plain java + a few "data only" classes (connector, jetty), large
> numbers of implementation classes together with some java (openejb), or
> the entire runtime object serialized (axis integration).
>
> So, there are 2 dimensions of incompatibility: GBeanInfo definitions
> and the classes of GBean attributes.  With care we might be able to
> make gbean infos upward compatible by only adding attributes and
> references and supplying in code default values when the new ones are
> missing.  Data only classes are perhaps not all that likely to change
> incompatibly if they are designed well enough to begin with.  The big
> problem comes when we are using large parts of the implementation as
> attributes.  It  might be worth considering if it would be practical to
> make more of the attributes data-like.
>
> >
> > The tricky part:
> >   - We, qualified in alphabetical order, ActiveMQ, Axis, Geronimo,
> > Jetty, OpenEJB, Tomcat, TranQL, et. al. cannot change our public API
> > *or* private Implementation of our APIs and still deserialize objects
> > from a previous version.
>
> I think what we can't change is the GBeanInfo definitions and the
> classes used as attributes.
>
> thanks
> david jencks
>
> >   - Unless we (the projecst above) agree not to bug fix, optimize,
> > refactor, repackage, or reorganize our code in a way that would break
> > deserialization of objects created with a previous versions of our
> > code, it will not be possible to upgrade applications leveraging these
> > projects without a redeploy.
> >
> >
> > PROJECT POLL:
> >   [ ]  We will do our best to ensure the implementations of our APIs
> > are serialization compatible to future versions of our code.
> >   [ ]  We do our best to ensure our public APIs, but use of our
> > implementations in such a way is not supported by us.
> >
> > USER POLL:
> >   [ ]  I do not mind redeploying my applications on each upgrade of
> > Geronimo et. al.
> >   [ ]  I do not want to start from scratch on each upgrade of Geronimo
> > et. al.
> >
> >
> >
> > -David
> >
> >
>
>


--
Davanum Srinivas - http://webservices.apache.org/~dims/
Reply | Threaded
Open this post in threaded view
|

[HUMOR] Re: [POLL] API and Implementation serialization compatibility and upgrading

Jeremy Boynes
In reply to this post by David Blevins
<humor>
Oh, for Pete's sake, this poll is about as Fair and Balanced as Fox News.

How about:

PROJECT POLL:
   [ ] We, ${projectName}, didn't really mean it when we said something
       was Serializable and feel free to change our APIs and
       implementation at any time without telling you.

   [ ] We do our best to ensure our public APIs, but use of them is not
       supported by us. Good luck guys.

USER POLL:
   [ ] I am a ${otherProject} user and quite like the incompatibilities
       between even minor dot releases - it gives my admins something to
       do. Please make sure ${projectName} behaves this way as well

   [ ] I believe the safest way to upgrade something is to wipe out
       everything that is there and start over every time. That way
       I don't get cooties from old software.

</humor>

OK, so it probably isn't that funny but let's at least have a serious
discussion about this.

--
Jeremy

David Blevins wrote:

> I really want to get some feedback from the developers, users, lurkers, other projects and everyone on this subject.  It shouldn't be a taboo to talk about or considered a "can of worms" or a "hot button."  It affects the project at pretty fundamental level, so I think it would be good if we did our best to get feedback.
>
> The problem:
>
>   Upgrading apps between versions of Geronimo and it's integrated projects is not currently possible without a redeploy.
>
> The facts:
>   - Our deploy system and storage of deployed apps is serialization based.
>   - At deploy time we build and Jetty web containers, Tomcat web containers, ActiveMQ queues/topics, OpenEJB ejb containers, Axis webservices, TranQL connectors and more
>   - We serialize those objects which include not just what they consider to be their public APIs but also the private implementations of those APIs.
>
> The tricky part:
>   - We, qualified in alphabetical order, ActiveMQ, Axis, Geronimo, Jetty, OpenEJB, Tomcat, TranQL, et. al. cannot change our public API *or* private Implementation of our APIs and still deserialize objects from a previous version.
>   - Unless we (the projecst above) agree not to bug fix, optimize, refactor, repackage, or reorganize our code in a way that would break deserialization of objects created with a previous versions of our code, it will not be possible to upgrade applications leveraging these projects without a redeploy.
>
>
> PROJECT POLL:
>   [ ]  We will do our best to ensure the implementations of our APIs are serialization compatible to future versions of our code.
>   [ ]  We do our best to ensure our public APIs, but use of our implementations in such a way is not supported by us.
>
> USER POLL:
>   [ ]  I do not mind redeploying my applications on each upgrade of Geronimo et. al.
>   [ ]  I do not want to start from scratch on each upgrade of Geronimo et. al.
>
>
>
> -David
>
>


Reply | Threaded
Open this post in threaded view
|

Re: [HUMOR] Re: [POLL] API and Implementation serialization compatibility and upgrading

Davanum Srinivas
Jeremy,

hehehe...problem is i just added some Serializable stuff in axis w/o
even thinking about it because Geronimo guys told me they needed it.
now, i can't change it :(

-- dims

On 5/14/05, Jeremy Boynes <[hidden email]> wrote:

> <humor>
> Oh, for Pete's sake, this poll is about as Fair and Balanced as Fox News.
>
> How about:
>
> PROJECT POLL:
>    [ ] We, ${projectName}, didn't really mean it when we said something
>        was Serializable and feel free to change our APIs and
>        implementation at any time without telling you.
>
>    [ ] We do our best to ensure our public APIs, but use of them is not
>        supported by us. Good luck guys.
>
> USER POLL:
>    [ ] I am a ${otherProject} user and quite like the incompatibilities
>        between even minor dot releases - it gives my admins something to
>        do. Please make sure ${projectName} behaves this way as well
>
>    [ ] I believe the safest way to upgrade something is to wipe out
>        everything that is there and start over every time. That way
>        I don't get cooties from old software.
>
> </humor>
>
> OK, so it probably isn't that funny but let's at least have a serious
> discussion about this.
>
> --
> Jeremy
>
> David Blevins wrote:
> > I really want to get some feedback from the developers, users, lurkers, other projects and everyone on this subject.  It shouldn't be a taboo to talk about or considered a "can of worms" or a "hot button."  It affects the project at pretty fundamental level, so I think it would be good if we did our best to get feedback.
> >
> > The problem:
> >
> >   Upgrading apps between versions of Geronimo and it's integrated projects is not currently possible without a redeploy.
> >
> > The facts:
> >   - Our deploy system and storage of deployed apps is serialization based.
> >   - At deploy time we build and Jetty web containers, Tomcat web containers, ActiveMQ queues/topics, OpenEJB ejb containers, Axis webservices, TranQL connectors and more
> >   - We serialize those objects which include not just what they consider to be their public APIs but also the private implementations of those APIs.
> >
> > The tricky part:
> >   - We, qualified in alphabetical order, ActiveMQ, Axis, Geronimo, Jetty, OpenEJB, Tomcat, TranQL, et. al. cannot change our public API *or* private Implementation of our APIs and still deserialize objects from a previous version.
> >   - Unless we (the projecst above) agree not to bug fix, optimize, refactor, repackage, or reorganize our code in a way that would break deserialization of objects created with a previous versions of our code, it will not be possible to upgrade applications leveraging these projects without a redeploy.
> >
> >
> > PROJECT POLL:
> >   [ ]  We will do our best to ensure the implementations of our APIs are serialization compatible to future versions of our code.
> >   [ ]  We do our best to ensure our public APIs, but use of our implementations in such a way is not supported by us.
> >
> > USER POLL:
> >   [ ]  I do not mind redeploying my applications on each upgrade of Geronimo et. al.
> >   [ ]  I do not want to start from scratch on each upgrade of Geronimo et. al.
> >
> >
> >
> > -David
> >
> >
>
>
>


--
Davanum Srinivas - http://webservices.apache.org/~dims/
Reply | Threaded
Open this post in threaded view
|

Re: [HUMOR] Re: [POLL] API and Implementation serialization compatibility and upgrading

Jeremy Boynes
Davanum Srinivas wrote:
> Jeremy,
>
> hehehe...problem is i just added some Serializable stuff in axis w/o
> even thinking about it because Geronimo guys told me they needed it.
> now, i can't change it :(
>

Muhahaha, that'll teach you to be helpful ;-)

I don't know what you changed but please remember that we are talking
about compatibility at the release level, not continuously, not at every
SNAPSHOT. My understanding of the Axis integration is that we are still
using an unstable build (a particular patch level) and so should not
expect, er, stability until an official release.

This is just normal software engineering - things change during
development but stabilize at release points - that's why we do releases.

--
Jeremy

Reply | Threaded
Open this post in threaded view
|

Re: [POLL] API and Implementation serialization compatibility and upgrading

Jeremy Boynes
In reply to this post by ammulder
Aaron Mulder wrote:

> I'd like some more information before answering this poll.  
> Namely, what exactly is serialized?  I guess I was under the impression
> that "configuration information" was serialized.  You make it sound like,
> for example, ActiveMQ itself is serialized.  Shouldn't we be able to
> serialize a bunch of settings (port=61616, active transport 1 =
> vm://localhost, active transport 2 = tcp://localhost:61616, ...) and apply
> those to version N+1 of ActiveMQ, so long as it didn't remove settings we
> use or add new required ones without defaults that we don't use?  Are we
> just not doing that?  I guess I need to take a closer look.
>
> Also, wearing my end user hat, I'm not as concerned about this
> issue as stated -- Every time I upgrade, for example, JBoss, I install it
> into a new directory, reapply any configuration changes, and redeploy my
> apps.  It might be different if the product offered an "upgrade in place"
> install option.  Still, I upgrade pretty rarely.  So an easy upgrade path
> is probably for me, at best, a "nice to have".
>

I posted those comments I promised and hope they clarify why this is a
big deal to me - basically, that massive deployment madates automated
"upgrade in place"

In reality, with the model you describe applications will always get
redeployed (and hence the config bundles rebuilt) in the new
configuration and none of the versioning issues apply.

> I'm more concerned about the inability to conveniently edit
> serialized configuration information, which I hope to take up on the other
> thread that Jeremy was going to start.  That is a lot more of a common
> issue for me -- more of a "must have".
>

Let's run through the use cases there as I think there are many options
for this.

--
Jeremy

Reply | Threaded
Open this post in threaded view
|

Re: [POLL] API and Implementation serialization compatibility and upgrading

Jeremy Boynes
In reply to this post by Phil Steitz-2
Phil Steitz wrote:

> Aaron Mulder wrote:
> <snip/>
>
>>
>>     Also, wearing my end user hat, I'm not as concerned about this
>> issue as stated -- Every time I upgrade, for example, JBoss, I install it
>> into a new directory, reapply any configuration changes, and redeploy my
>> apps.  It might be different if the product offered an "upgrade in place"
>> install option.  Still, I upgrade pretty rarely.  So an easy upgrade
>> path is probably for me, at best, a "nice to have".
>>
>>     I'm more concerned about the inability to conveniently edit
>> serialized configuration information, which I hope to take up on the
>> other thread that Jeremy was going to start.  That is a lot more of a
>> common issue for me -- more of a "must have".
>>
>> Aaron
>
>
> As a user, I pretty much agree with Aaron (both points), with the
> following additional comment on the first one:
>
> Major version upgrades are relatively rare events, but applying patches
> / service packs / whatever they may be called is more common and
> sometimes needs to be done quickly with minimal disruption.  Having to
> redeploy all apps on a server to which a patch is applied is not ideal.
> Having strange things happen after patches are applied is even less fun ;-)
>
> So I guess it all depends on what exactly we mean by "upgrade"...
>

I put my thoughts on the wiki:
http://wiki.apache.org/geronimo/Architecture/ConfigurationManagement

My expectation is that if the original server and a "patched" server can
both provide the environment the application expects then it will run
identically on both systems.

I would expect the change process to be something like:
* get patched version of server bundle from somewhere
* install the patched bundle in a test server
* check the existing application still runs and the problem is fixed
* build a new application bundle depending on the patched server version

you then have a couple of choices for rollout, either
* push patched server bundle to production
* pro-actively push to runtime servers, replacing the buggy server

or
* push patched server bundle to production
* push updated application bundle to production
* roll new application bundle to runtime servers as appropriate
* each runtime server will get the patched server bundle from
   config control so that it can support the new application version

and other variations of either approach.

--
Jeremy

Reply | Threaded
Open this post in threaded view
|

Re: [HUMOR] Re: [POLL] API and Implementation serialization compatibility and upgrading

David Blevins
In reply to this post by Jeremy Boynes
On Sat, May 14, 2005 at 09:50:45PM -0700, Jeremy Boynes wrote:

> Davanum Srinivas wrote:
> >Jeremy,
> >
> >hehehe...problem is i just added some Serializable stuff in axis w/o
> >even thinking about it because Geronimo guys told me they needed it.
> >now, i can't change it :(
> >
>
> Muhahaha, that'll teach you to be helpful ;-)
>
> I don't know what you changed but please remember that we are talking
> about compatibility at the release level, not continuously, not at every
> SNAPSHOT.

We are talking about upgrading or patching or anything that results in changed class files.  Changed class files means risk of not being able to deserialize data from previous class files.

-David
Reply | Threaded
Open this post in threaded view
|

Re: [HUMOR] Re: [POLL] API and Implementation serialization compatibility and upgrading

David Blevins
In reply to this post by Jeremy Boynes
Nice spin.  Though you're joking around, I'll answer seriously.

PROJECTS:
Again we are talking about changed class files, which can result from a patch or upgrade.  When people make things serializable, they are only guaranteeing that data serialized from that class and be deserialized back into that exact class.  They are not making claims of past or future versions of that class.

USERS:
Users do care about incompatibilities between minor dot and patch releases.  But users do not care about compatibility of private class files between minor dot and patch releases.  But as the storage format for configuration data is public and private class files, we have made this their problem at upgrade/patch time.

-David

On Sat, May 14, 2005 at 08:14:49PM -0700, Jeremy Boynes wrote:

> <humor>
> Oh, for Pete's sake, this poll is about as Fair and Balanced as Fox News.
>
> How about:
>
> PROJECT POLL:
>   [ ] We, ${projectName}, didn't really mean it when we said something
>       was Serializable and feel free to change our APIs and
>       implementation at any time without telling you.
>
>   [ ] We do our best to ensure our public APIs, but use of them is not
>       supported by us. Good luck guys.
>
> USER POLL:
>   [ ] I am a ${otherProject} user and quite like the incompatibilities
>       between even minor dot releases - it gives my admins something to
>       do. Please make sure ${projectName} behaves this way as well
>
>   [ ] I believe the safest way to upgrade something is to wipe out
>       everything that is there and start over every time. That way
>       I don't get cooties from old software.
>
> </humor>
>
> OK, so it probably isn't that funny but let's at least have a serious
> discussion about this.
>
> --
> Jeremy
>
> David Blevins wrote:
> >I really want to get some feedback from the developers, users, lurkers,
> >other projects and everyone on this subject.  It shouldn't be a taboo to
> >talk about or considered a "can of worms" or a "hot button."  It affects
> >the project at pretty fundamental level, so I think it would be good if we
> >did our best to get feedback.
> >
> >The problem:
> >
> >  Upgrading apps between versions of Geronimo and it's integrated projects
> >  is not currently possible without a redeploy.
> >
> >The facts:
> >  - Our deploy system and storage of deployed apps is serialization based.
> >  - At deploy time we build and Jetty web containers, Tomcat web
> >  containers, ActiveMQ queues/topics, OpenEJB ejb containers, Axis
> >  webservices, TranQL connectors and more
> >  - We serialize those objects which include not just what they consider
> >  to be their public APIs but also the private implementations of those
> >  APIs.
> >
> >The tricky part:
> >  - We, qualified in alphabetical order, ActiveMQ, Axis, Geronimo, Jetty,
> >  OpenEJB, Tomcat, TranQL, et. al. cannot change our public API *or*
> >  private Implementation of our APIs and still deserialize objects from a
> >  previous version.
> >  - Unless we (the projecst above) agree not to bug fix, optimize,
> >  refactor, repackage, or reorganize our code in a way that would break
> >  deserialization of objects created with a previous versions of our code,
> >  it will not be possible to upgrade applications leveraging these
> >  projects without a redeploy.
> >
> >
> >PROJECT POLL:
> >  [ ]  We will do our best to ensure the implementations of our APIs are
> >  serialization compatible to future versions of our code.
> >  [ ]  We do our best to ensure our public APIs, but use of our
> >  implementations in such a way is not supported by us.
> >
> >USER POLL:
> >  [ ]  I do not mind redeploying my applications on each upgrade of
> >  Geronimo et. al.
> >  [ ]  I do not want to start from scratch on each upgrade of Geronimo et.
> >  al.
> >
> >
> >
> >-David
> >
> >
>

 LocalWords:  Boynes
Reply | Threaded
Open this post in threaded view
|

Re: [POLL] API and Implementation serialization compatibility and upgrading

Hiram Chirino
In reply to this post by David Blevins
Hi,

 From the ActiveMQ viewpoint,  we would rather:

> PROJECT POLL:
>   [ ]  We will do our best to ensure the implementations of our APIs
> are serialization compatible to future versions of our code.
>   [X]  We do our best to ensure our public APIs, but use of our
> implementations in such a way is not supported by us.

ActiveMQ standalone does not currently have these serialization issues
so it's unfortunate that for us to play nice with Geronimo we would
have to add on a Serialization requirement on our GBeans.  It's
actually quite unfortunate that ActiveMQ even needs to implement
GBeans.  It just proves that the geronimo IOC container is not
transparent.  And adding the Serialization requirements makes that even
worse.

Reply | Threaded
Open this post in threaded view
|

Re: [HUMOR] Re: [POLL] API and Implementation serialization compatibility and upgrading

Sanjiva Weerawarana
In reply to this post by David Blevins
What's the reason for originally deciding to go with object
serialization to store configuration *data* instead of some text format,
say an XML format?

Sanjiva.

On Mon, 2005-05-16 at 08:44 -0700, David Blevins wrote:

> Nice spin.  Though you're joking around, I'll answer seriously.
>
> PROJECTS:
> Again we are talking about changed class files, which can result from a patch or upgrade.  When people make things serializable, they are only guaranteeing that data serialized from that class and be deserialized back into that exact class.  They are not making claims of past or future versions of that class.
>
> USERS:
> Users do care about incompatibilities between minor dot and patch releases.  But users do not care about compatibility of private class files between minor dot and patch releases.  But as the storage format for configuration data is public and private class files, we have made this their problem at upgrade/patch time.
>
> -David
>
> On Sat, May 14, 2005 at 08:14:49PM -0700, Jeremy Boynes wrote:
> > <humor>
> > Oh, for Pete's sake, this poll is about as Fair and Balanced as Fox News.
> >
> > How about:
> >
> > PROJECT POLL:
> >   [ ] We, ${projectName}, didn't really mean it when we said something
> >       was Serializable and feel free to change our APIs and
> >       implementation at any time without telling you.
> >
> >   [ ] We do our best to ensure our public APIs, but use of them is not
> >       supported by us. Good luck guys.
> >
> > USER POLL:
> >   [ ] I am a ${otherProject} user and quite like the incompatibilities
> >       between even minor dot releases - it gives my admins something to
> >       do. Please make sure ${projectName} behaves this way as well
> >
> >   [ ] I believe the safest way to upgrade something is to wipe out
> >       everything that is there and start over every time. That way
> >       I don't get cooties from old software.
> >
> > </humor>
> >
> > OK, so it probably isn't that funny but let's at least have a serious
> > discussion about this.
> >
> > --
> > Jeremy
> >
> > David Blevins wrote:
> > >I really want to get some feedback from the developers, users, lurkers,
> > >other projects and everyone on this subject.  It shouldn't be a taboo to
> > >talk about or considered a "can of worms" or a "hot button."  It affects
> > >the project at pretty fundamental level, so I think it would be good if we
> > >did our best to get feedback.
> > >
> > >The problem:
> > >
> > >  Upgrading apps between versions of Geronimo and it's integrated projects
> > >  is not currently possible without a redeploy.
> > >
> > >The facts:
> > >  - Our deploy system and storage of deployed apps is serialization based.
> > >  - At deploy time we build and Jetty web containers, Tomcat web
> > >  containers, ActiveMQ queues/topics, OpenEJB ejb containers, Axis
> > >  webservices, TranQL connectors and more
> > >  - We serialize those objects which include not just what they consider
> > >  to be their public APIs but also the private implementations of those
> > >  APIs.
> > >
> > >The tricky part:
> > >  - We, qualified in alphabetical order, ActiveMQ, Axis, Geronimo, Jetty,
> > >  OpenEJB, Tomcat, TranQL, et. al. cannot change our public API *or*
> > >  private Implementation of our APIs and still deserialize objects from a
> > >  previous version.
> > >  - Unless we (the projecst above) agree not to bug fix, optimize,
> > >  refactor, repackage, or reorganize our code in a way that would break
> > >  deserialization of objects created with a previous versions of our code,
> > >  it will not be possible to upgrade applications leveraging these
> > >  projects without a redeploy.
> > >
> > >
> > >PROJECT POLL:
> > >  [ ]  We will do our best to ensure the implementations of our APIs are
> > >  serialization compatible to future versions of our code.
> > >  [ ]  We do our best to ensure our public APIs, but use of our
> > >  implementations in such a way is not supported by us.
> > >
> > >USER POLL:
> > >  [ ]  I do not mind redeploying my applications on each upgrade of
> > >  Geronimo et. al.
> > >  [ ]  I do not want to start from scratch on each upgrade of Geronimo et.
> > >  al.
> > >
> > >
> > >
> > >-David
> > >
> > >
> >
>
>  LocalWords:  Boynes
>

Reply | Threaded
Open this post in threaded view
|

Re: [POLL] API and Implementation serialization compatibility and upgrading

David Jencks
In reply to this post by Hiram Chirino
Slow down a minute.

I just looked at the activemq gbeans used in geronimo currently and
they appear to use for attributes only types from java.xxx.  As I
already indicated in a previous post, the possible problems come from
changing classes used as attributes and changing the GBeanInfo classes
themselves.  An incompatible change in GBeanInfo classes is very likely
to require redeployment of everything and redesign of the descriptors.  
Since activemq is only using standard java types as gbean attributes,
activemq is free to change all their classes as they see fit and make
none of them serializable as long as the types and names of the gbean
attributes remain the same.

As far as the form of metadata needed for an IOC container, I currently
prefer GBeanInfo to the alternatives I am aware of.  I think it's
important to specify exactly what interface is presented to the
container, which IIUC spring does not require, and after working a lot
with the jboss xmlbean implementation and writing the xdoclet xmlbeans
plugin, I think that xml is perhaps the worst choice possible, and
javadoc tags not much better.


thanks
david jencks

On May 16, 2005, at 4:13 PM, Hiram Chirino wrote:

> Hi,
>
> From the ActiveMQ viewpoint,  we would rather:
>
>> PROJECT POLL:
>>   [ ]  We will do our best to ensure the implementations of our APIs
>> are serialization compatible to future versions of our code.
>>   [X]  We do our best to ensure our public APIs, but use of our
>> implementations in such a way is not supported by us.
>
> ActiveMQ standalone does not currently have these serialization issues
> so it's unfortunate that for us to play nice with Geronimo we would
> have to add on a Serialization requirement on our GBeans.  It's
> actually quite unfortunate that ActiveMQ even needs to implement
> GBeans.  It just proves that the geronimo IOC container is not
> transparent.  And adding the Serialization requirements makes that
> even worse.
>

Reply | Threaded
Open this post in threaded view
|

Re: [POLL] API and Implementation serialization compatibility and upgrading

Hiram Chirino
Hi David,

Yes you are right.  Our current activemq broker configuration is a bit  
simplistic.  I wish it was as easy to support complex broker  
configuration in geronimo as it is in spring.  With spring we can  
support a really rich broker configuration language see:  
http://activemq.codehaus.org/checkout/activemq/modules/core/src/conf/ 
org/activemq/activemq.dtd

The question is what's the simplest way to do something similar with  
geronimo?  I tried using the GBean approach and it took down a route  
that seemed like I was going to have to create gbean wrapper classes  
for each activemq component.  I was hoping for something more  
transparent since the ActiveMQ components are very IOCish I don't  
really see the need for extra gbean wrappers.

Regards,
Hiram

On May 17, 2005, at 2:39 PM, David Jencks wrote:

> Slow down a minute.
>
> I just looked at the activemq gbeans used in geronimo currently and  
> they appear to use for attributes only types from java.xxx.  As I  
> already indicated in a previous post, the possible problems come from  
> changing classes used as attributes and changing the GBeanInfo classes  
> themselves.  An incompatible change in GBeanInfo classes is very  
> likely to require redeployment of everything and redesign of the  
> descriptors.  Since activemq is only using standard java types as  
> gbean attributes, activemq is free to change all their classes as they  
> see fit and make none of them serializable as long as the types and  
> names of the gbean attributes remain the same.
>
> As far as the form of metadata needed for an IOC container, I  
> currently prefer GBeanInfo to the alternatives I am aware of.  I think  
> it's important to specify exactly what interface is presented to the  
> container, which IIUC spring does not require, and after working a lot  
> with the jboss xmlbean implementation and writing the xdoclet xmlbeans  
> plugin, I think that xml is perhaps the worst choice possible, and  
> javadoc tags not much better.
>
>
> thanks
> david jencks
>
> On May 16, 2005, at 4:13 PM, Hiram Chirino wrote:
>
>> Hi,
>>
>> From the ActiveMQ viewpoint,  we would rather:
>>
>>> PROJECT POLL:
>>>   [ ]  We will do our best to ensure the implementations of our APIs  
>>> are serialization compatible to future versions of our code.
>>>   [X]  We do our best to ensure our public APIs, but use of our  
>>> implementations in such a way is not supported by us.
>>
>> ActiveMQ standalone does not currently have these serialization  
>> issues so it's unfortunate that for us to play nice with Geronimo we  
>> would have to add on a Serialization requirement on our GBeans.  It's  
>> actually quite unfortunate that ActiveMQ even needs to implement  
>> GBeans.  It just proves that the geronimo IOC container is not  
>> transparent.  And adding the Serialization requirements makes that  
>> even worse.
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: [HUMOR] Re: [POLL] API and Implementation serialization compatibility and upgrading

Jeremy Boynes-2
In reply to this post by Sanjiva Weerawarana
Sanjiva Weerawarana wrote:
> What's the reason for originally deciding to go with object
> serialization to store configuration *data* instead of some text format,
> say an XML format?
>

This list having horrendous email delays at the moment. I hope this is
covered in another thread.

--
Jeremy

Reply | Threaded
Open this post in threaded view
|

Re: [POLL] API and Implementation serialization compatibility and upgrading

Dain Sundstrom
In reply to this post by David Jencks
On May 17, 2005, at 11:39 AM, David Jencks wrote:

> As far as the form of metadata needed for an IOC container, I  
> currently prefer GBeanInfo to the alternatives I am aware of.  I  
> think it's important to specify exactly what interface is presented  
> to the container, which IIUC spring does not require, and after  
> working a lot with the jboss xmlbean implementation and writing the  
> xdoclet xmlbeans plugin, I think that xml is perhaps the worst  
> choice possible, and javadoc tags not much better.

Why not support all three?  We could provide a simple pluggable  
interface to get the required metadata from store, be it GBeanInfo,  
xml, annotations, etc.

-dain
Reply | Threaded
Open this post in threaded view
|

Re: [POLL] API and Implementation serialization compatibility and upgrading

Jeremy Boynes
In reply to this post by Hiram Chirino
Sorry if this has already been answered but with all the mail delays ...


Hiram Chirino wrote:

> Hi David,
>
> Yes you are right.  Our current activemq broker configuration is a bit  
> simplistic.  I wish it was as easy to support complex broker  
> configuration in geronimo as it is in spring.  With spring we can  
> support a really rich broker configuration language see:  
> http://activemq.codehaus.org/checkout/activemq/modules/core/src/conf/ 
> org/activemq/activemq.dtd
>
> The question is what's the simplest way to do something similar with  
> geronimo?  I tried using the GBean approach and it took down a route  
> that seemed like I was going to have to create gbean wrapper classes  
> for each activemq component.  I was hoping for something more  
> transparent since the ActiveMQ components are very IOCish I don't  
> really see the need for extra gbean wrappers.
>

I am guessing that activemq-to-spring.xsl is converting an activemq
instance document described by the DTD into a Spring configuration file,
which Spring is then able to use to instantiate the broker components.
Is this right? (If not ignore the rest of this mail).

If this is true, it seems plausible that you could provide an
activemq-to-gbean.xsl that converted the instance document into a GBean
service configuration file containing multiple GBean definitions linked
together with GBean references and configured with attribute values.
Given you have a DTD I would assume the values would be simple types and
hence would not run into serialization problems.

You could also implement this as a Geronimo ConfigurationBuilder that
took an activemq instance document and converted it directly to a
Configuration which could then be directly installed into any server.

I'm sure you tried that and there must be some problem I can't see on
the surface - could you give us a bit more information on where it
wasn't working?

--
Jeremy

Reply | Threaded
Open this post in threaded view
|

Re: [POLL] API and Implementation serialization compatibility and upgrading

Jeremy Boynes
In reply to this post by Dain Sundstrom
Beware email delays :-(

Dain Sundstrom wrote:

> On May 17, 2005, at 11:39 AM, David Jencks wrote:
>
>> As far as the form of metadata needed for an IOC container, I  
>> currently prefer GBeanInfo to the alternatives I am aware of.  I  
>> think it's important to specify exactly what interface is presented  
>> to the container, which IIUC spring does not require, and after  
>> working a lot with the jboss xmlbean implementation and writing the  
>> xdoclet xmlbeans plugin, I think that xml is perhaps the worst  choice
>> possible, and javadoc tags not much better.
>
>
> Why not support all three?  We could provide a simple pluggable  
> interface to get the required metadata from store, be it GBeanInfo,  
> xml, annotations, etc.
>

Sure - as you know we designed the GBeanInfo so that it could easily be
constructed from annotations when they were available (somewhere around
4/24/05 on OSX IIRC). And I thought Hiram already had an XML to
GBeanInfo generator.

GBeanInfo is, of course, "class" type metadata - it describes what
attributes, references, etc. a GBean class has and not the values for
any specific instance. That is why a static value (in JDK1.4) or
annotations (in JDK1.5) are both good solutions as the GBeanInfo is
fixed at the same time the class is compiled.

When the Configuration is built, its code dependencies tie it to a
specific version of the class files in use (e.g. those from foo-1.0.jar)
so the instance data will be reloaded back into compatible classes.
Matching the configuration instance to a codebase that can be used to
instantiate it is the ConfigurationManager's job.

--
Jeremy

12