[RFF] Patch tracking/metrics

[Guilherme Salgado]

Hi folks,

I've been working on this app[1] (based on Patchwork) to track patches
submitted/accepted upstream by Linaro engineers. It's still a work in
progress and we're waiting for IS to deploy it but I wanted to show you
what I have so far and ask for feedback, so I've deployed it on an ec2
instance:

http://ec2-184-73-78-92.compute-1.amazonaws.com/

As you'll notice, that's the same front page you get on a regular
Patchwork instance, which allows you to browse the patches of every
project. You probably won't have to use that often, as everything you
need to deal with should be on the page below:

http://ec2-184-73-78-92.compute-1.amazonaws.com/user/submitted/

This one contains all patches you submitted that haven't received
feedback yet. We want that to reflect reality to make sure our metrics
are accurate, so we already have a script to mark the ones that are
committed upstream and may use a similar approach to track
rejected/superseded ones, but for now we'll need to manually mark the
rejected/superseded ones.

The script mentioned above will scan a git repo for each project and
update the state of patches that are found to be committed there, but
it's not running yet as first I need to know the git (http works too)
URL of every project listed there. if you know it for any of them,
please do let me know.

Finally, the page below has some basic metrics to demo how we'll be
using this data

http://ec2-184-73-78-92.compute-1.amazonaws.com/metrics/


Some things to notice:

- This is a temporary deployment, so don't bother making changes (other
than those when playing around) as those will probably be lost

- Login via OpenID using the Launchpad login service, so there's no need
to register

- Not all of your patches may be shown under /user/submitted. if that's
the case, make sure all your email addresses are registered in Launchpad
as we're sucking that data from there periodically and the next time we
do so we'll merge all your email addresses under a single user

- Some numbers in /metrics are skewed because of patches that have
multiple versions; you'll be asked to flag superseded versions so that
they're not included in the counts, and you're encouraged to do so for
some of them to see how it works, but given that changes will be lost,
don't bother doing it for more than a few

- The other-unknown project is where we put patches for which we can't
guess a project.
- in some cases this is because the patches are sent only to
***@linaro.org so they need to be manually moved to the
appropriate project (there's a form at the bottom of every patch
list which allow you to do that).
- in other cases it's because we're missing a project in patchwork. if
that's the case we can register that project and there's a script
which runs periodically and will move these patches to the new
project


[1] https://wiki.linaro.org/Platform/Infrastructure/Specs/PatchTracking

[Guilherme Salgado]

Post by Guilherme Salgado
- Not all of your patches may be shown under /user/submitted. if that's
the case, make sure all your email addresses are registered in Launchpad
as we're sucking that data from there periodically and the next time we
do so we'll merge all your email addresses under a single user

This doesn't seem to be working: I have (and have had for a long time)
but when I sign into this patchwork via openid it only knows about
the chiark one, not the linaro one.

That's because Launchpad only sends us your preferred email address when
you login via OpenID. The next time we suck user data from Launchpad,
it will notice these other email addresses can be linked to your
Patchwork account and will do so.

I've triggered a manual run now, so you'll see the extra email addresses
there shortly.

[Peter Maydell]

Post by Guilherme Salgado

but when I sign into this patchwork via openid it only knows about
the chiark one, not the linaro one.

That's because Launchpad only sends us your preferred email address when
you login via OpenID.  The next time we suck user data from Launchpad,
it will notice these other email addresses

Oh, of course, it only knows it needs to get your email addresses
when you've logged into it. Sorry for the confusion.

-- PMM

[James Westby]

Hi,

This service is going to be used by management to get an idea of the
number of patches going to each project over time, the number of patches
submitted upstream by each team over time, the % of patches accepted
upstream, the average time from submission to acceptance for each
project, and other things like that.

For this to work well, and for your work to be accurately counted, you
are going to be expected to keep it up to date as you submit patches
upstream.

Now is the time to speak up if the interface doesn't have what you need
to do this. Obviously it would be better if it took care of everything
for you, but we don't think that everything can be automated. If you
know better then we would obviously like to hear about that too.

Thanks,

James

Post by Guilherme Salgado
Hi folks,
I've been working on this app[1] (based on Patchwork) to track patches
submitted/accepted upstream by Linaro engineers. It's still a work in
progress and we're waiting for IS to deploy it but I wanted to show you
what I have so far and ask for feedback, so I've deployed it on an ec2
http://ec2-184-73-78-92.compute-1.amazonaws.com/
As you'll notice, that's the same front page you get on a regular
Patchwork instance, which allows you to browse the patches of every
project. You probably won't have to use that often, as everything you
http://ec2-184-73-78-92.compute-1.amazonaws.com/user/submitted/
This one contains all patches you submitted that haven't received
feedback yet. We want that to reflect reality to make sure our metrics
are accurate, so we already have a script to mark the ones that are
committed upstream and may use a similar approach to track
rejected/superseded ones, but for now we'll need to manually mark the
rejected/superseded ones.
The script mentioned above will scan a git repo for each project and
update the state of patches that are found to be committed there, but
it's not running yet as first I need to know the git (http works too)
URL of every project listed there. if you know it for any of them,
please do let me know.
Finally, the page below has some basic metrics to demo how we'll be
using this data
http://ec2-184-73-78-92.compute-1.amazonaws.com/metrics/
- This is a temporary deployment, so don't bother making changes (other
than those when playing around) as those will probably be lost
- Login via OpenID using the Launchpad login service, so there's no need
to register
- Not all of your patches may be shown under /user/submitted. if that's
the case, make sure all your email addresses are registered in Launchpad
as we're sucking that data from there periodically and the next time we
do so we'll merge all your email addresses under a single user
- Some numbers in /metrics are skewed because of patches that have
multiple versions; you'll be asked to flag superseded versions so that
they're not included in the counts, and you're encouraged to do so for
some of them to see how it works, but given that changes will be lost,
don't bother doing it for more than a few
- The other-unknown project is where we put patches for which we can't
guess a project.
- in some cases this is because the patches are sent only to
appropriate project (there's a form at the bottom of every patch
list which allow you to do that).
- in other cases it's because we're missing a project in patchwork. if
that's the case we can register that project and there's a script
which runs periodically and will move these patches to the new
project
[1] https://wiki.linaro.org/Platform/Infrastructure/Specs/PatchTracking
--
Guilherme Salgado <https://launchpad.net/~salgado>

Non-text part: application/pgp-signature

Post by Guilherme Salgado
_______________________________________________
linaro-dev mailing list
http://lists.linaro.org/mailman/listinfo/linaro-dev

[Alexander Sack]

Post by James Westby
Hi,
This service is going to be used by management to get an idea of the
number of patches going to each project over time, the number of patches
submitted upstream by each team over time, the % of patches accepted
upstream, the average time from submission to acceptance for each
project, and other things like that.
For this to work well, and for your work to be accurately counted, you
are going to be expected to keep it up to date as you submit patches
upstream.
Now is the time to speak up if the interface doesn't have what you need
to do this. Obviously it would be better if it took care of everything
for you, but we don't think that everything can be automated. If you
know better then we would obviously like to hear about that too.

Wonder, does this test instance really need to use authentication of
launchpad staging? Just noticed that, because it neither logged me in
automatically nor where my credentials properly filled out in my
browser.

[Martin Pool]

Post by Alexander Sack

Post by James Westby
Now is the time to speak up if the interface doesn't have what you need
to do this. Obviously it would be better if it took care of everything
for you, but we don't think that everything can be automated. If you
know better then we would obviously like to hear about that too.

Wonder, does this test instance really need to use authentication of
launchpad staging? Just noticed that, because it neither logged me in
automatically nor where my credentials properly filled out in my
browser.

I haven't looked at the code, but that might be caused by
<https://bugs.launchpad.net/launchpadlib/+bug/714043>.

[Guilherme Salgado]

Post by Alexander Sack

Post by James Westby
Hi,
This service is going to be used by management to get an idea of the
number of patches going to each project over time, the number of patches
submitted upstream by each team over time, the % of patches accepted
upstream, the average time from submission to acceptance for each
project, and other things like that.
For this to work well, and for your work to be accurately counted, you
are going to be expected to keep it up to date as you submit patches
upstream.
Now is the time to speak up if the interface doesn't have what you need
to do this. Obviously it would be better if it took care of everything
for you, but we don't think that everything can be automated. If you
know better then we would obviously like to hear about that too.

Wonder, does this test instance really need to use authentication of
launchpad staging? Just noticed that, because it neither logged me in
automatically nor where my credentials properly filled out in my
browser.

In order to have patches associated with the logged in user, we need the
user's email address, which is only sent by the Launchpad Login Service
to certain authorized consumer sites. I didn't want to add that ec2
host as an authorized site to login.lp.net so I chose to use
login.staging.lp.net instead.

[Peter Maydell]

Post by James Westby
This service is going to be used by management to get an idea of the
number of patches going to each project over time, the number of patches
submitted upstream by each team over time, the % of patches accepted
upstream, the average time from submission to acceptance for each
project, and other things like that.
For this to work well, and for your work to be accurately counted, you
are going to be expected to keep it up to date as you submit patches
upstream.

Hmm, that's higher-maintenance than the original "just cc this email
address" proposal. In that case I'd like it to be able to replace my
manual wiki-based qemu patches page:
https://wiki.linaro.org/PeterMaydell/QemuPatchStatus

Missing features for that:
* ability to deal with whole patchsets as a single 'line item'
in patchwork, so you can move a 10-patch patchset from state to
state without it being a huge amount of drudgery
For bonus marks, patchwork ought to keep the 'cover letter' email
which is the one which actually ties the patchset together and
gives the rationale...
* missing statuses for tracking patch progress; at the moment
patch lifecycle for me is:
'waiting for review'
'will put into next pull request'
'pull request sent, not committed'
'committed'
not all of which have patchwork statuses. There's also
'picked up by another submaintainer but not committed yet'.
I guess I could bodge some of the existing ones like 'awaiting
upstream' for at least one of these.

Things that would be nice:
* you ought to be able to change patch states from the top level
list by ticking ticky boxes; at the moment it looks like you have
to go into the individual patch page to do this. Combined with the
lack of any sensible handling of patchsets, this means that marking
a patchset as applied is just way too much work.
* you might want some sort of way to say "this patchset is version 2
of that one", otherwise your statistics are going to be a bit weird
if you think all the patches in v1 are "this was never accepted" and
the patches in v2 have a time-to-commit starting from when v2 was
posted rather than from when v1 was posted.
* ability to add other people's patches (ie by non-Linaro
people) to my "need to review this patch" list and to "will put into
next pull request", etc. [I know this is a bit out of scope for
linaro's metrics tracking, but I definitely don't want to have to
track patches in more than one place if I can avoid it]

On the subject of patch tracking, should I cc patches@ for pull
requests (ie when I ask upstream to commit things)? I'm guessing
not since patches@ has already seen all the patches on first submission
and doesn't need them again.

PS: upstream's git URL for the benefit of automatic 'this patch
got committed' script: git://git.qemu.org/qemu.git

thanks
-- PMM

[James Westby]

Post by Peter Maydell

Post by James Westby
This service is going to be used by management to get an idea of the
number of patches going to each project over time, the number of patches
submitted upstream by each team over time, the % of patches accepted
upstream, the average time from submission to acceptance for each
project, and other things like that.
For this to work well, and for your work to be accurately counted, you
are going to be expected to keep it up to date as you submit patches
upstream.

Hmm, that's higher-maintenance than the original "just cc this email
address" proposal.

Yes, but a system that only has that information can never provide all
of the statistics that I outlined above.

Post by Peter Maydell
In that case I'd like it to be able to replace my
https://wiki.linaro.org/PeterMaydell/QemuPatchStatus
* ability to deal with whole patchsets as a single 'line item'
in patchwork, so you can move a 10-patch patchset from state to
state without it being a huge amount of drudgery

It can do this. It currently requires you to click to link the patches
from the set, but we've discussed ways to do this automatically in most
cases. I'm not sure if we have specific plans to improve this currently
though.

Post by Peter Maydell
For bonus marks, patchwork ought to keep the 'cover letter' email
which is the one which actually ties the patchset together and
gives the rationale...

I can't say whether it does that or not, Guilherme?

Post by Peter Maydell
* missing statuses for tracking patch progress; at the moment
'waiting for review'
'will put into next pull request'
'pull request sent, not committed'

It's not clear to me what these statuses mean? The patch is reviewed,
and once it is accepted it then goes in to a pull request and is
committed that way?

Post by Peter Maydell
'committed'
not all of which have patchwork statuses. There's also
'picked up by another submaintainer but not committed yet'.
I guess I could bodge some of the existing ones like 'awaiting
upstream' for at least one of these.

Yeah, it sounds like an "approved but not committed" state that would
apply to most projects.

If it is important for you to differentiate then we could add the extra
states and map them to that state for reporting.

Post by Peter Maydell
* you ought to be able to change patch states from the top level
list by ticking ticky boxes; at the moment it looks like you have
to go into the individual patch page to do this. Combined with the
lack of any sensible handling of patchsets, this means that marking
a patchset as applied is just way too much work.

I believe that if you go to your page listing outstanding patches you
can bulk-edit. I agree that bulk editing is an important feature.

Post by Peter Maydell
* you might want some sort of way to say "this patchset is version 2
of that one", otherwise your statistics are going to be a bit weird
if you think all the patches in v1 are "this was never accepted" and
the patches in v2 have a time-to-commit starting from when v2 was
posted rather than from when v1 was posted.

I believe this is possible, but I'm not sure how it is done. Guilherme?

I agree about the need for care about reporting times when this happens.

Post by Peter Maydell
* ability to add other people's patches (ie by non-Linaro
people) to my "need to review this patch" list and to "will put into
next pull request", etc. [I know this is a bit out of scope for
linaro's metrics tracking, but I definitely don't want to have to
track patches in more than one place if I can avoid it]

Yes, I think this is out of scope for the current project. I don't know
if we can do this very cheaply for you though, as I would agree that one
place to do all this would be good.

Post by Peter Maydell
requests (ie when I ask upstream to commit things)? I'm guessing
and doesn't need them again.

It sounds like there is no need. I'm also guessing that if you have all
patches reviewed first there isn't a long time between pull request and
pull?

Thanks,

James

[Peter Maydell]

Post by James Westby

Post by Peter Maydell

Post by James Westby
This service is going to be used by management to get an idea of the
number of patches going to each project over time, the number of patches
submitted upstream by each team over time, the % of patches accepted
upstream, the average time from submission to acceptance for each
project, and other things like that.
For this to work well, and for your work to be accurately counted, you
are going to be expected to keep it up to date as you submit patches
upstream.

Hmm, that's higher-maintenance than the original "just cc this email
address" proposal.

Yes, but a system that only has that information can never provide all
of the statistics that I outlined above.

Sure. It does mean that fixing deficiencies in the system is more
important than if it's just collecting patches and only needs to
be interacted with by a few people.

Post by James Westby

Post by Peter Maydell
In that case I'd like it to be able to replace my
https://wiki.linaro.org/PeterMaydell/QemuPatchStatus
 * ability to deal with whole patchsets as a single 'line item'
   in patchwork, so you can move a 10-patch patchset from state to
   state without it being a huge amount of drudgery

It can do this. It currently requires you to click to link the patches
from the set, but we've discussed ways to do this automatically in most
cases. I'm not sure if we have specific plans to improve this currently
though.

It's a feature I'd really like to see, and upstream sounded
receptive to it:
http://lists.ozlabs.org/pipermail/patchwork/2011-January/000348.html

so it would be great if we had the resources to do something here.

Post by James Westby

Post by Peter Maydell
 * missing statuses for tracking patch progress; at the moment
   'waiting for review'
   'will put into next pull request'
   'pull request sent, not committed'

It's not clear to me what these statuses mean? The patch is reviewed,
and once it is accepted it then goes in to a pull request and is
committed that way?

Basically, some qemu patches I send just get committed, which
is the straightforward case. Some patches don't get any review
comments of any form, so they "time out" and I eventually try
to batch them up and resubmit them via a pull request. I want
to be able to distinguish the 'timed out' patches from the others.

So "waiting for review" means "patch has gone on list and hasn't
had any comments"; "will put into next pull req" means "patch has
gone on list, had no comments but it's been a couple of weeks
so I consider it to have passed review by default"; "pull request
sent" means "I've sent upstream a pull request email but am waiting
for upstream to actually pull (or deny, as the case may be)".
"committed" means "actually got into upstream's git tree".

I'm not wedded to this particular set of states but it might
be nice for my purposes to have a little more flexibility.
I'll have a go with the existing set of states for a bit and
that ought to give me a better idea of whether I really need more.

Post by James Westby

Post by Peter Maydell
 * you ought to be able to change patch states from the top level
   list by ticking ticky boxes; at the moment it looks like you have
   to go into the individual patch page to do this. Combined with the
   lack of any sensible handling of patchsets, this means that marking
   a patchset as applied is just way too much work.

I believe that if you go to your page listing outstanding patches you
can bulk-edit. I agree that bulk editing is an important feature.

Looks like you can, yes. So not being able to do that from the
project page is just a UI inconsistency.

Post by James Westby

Post by Peter Maydell
 * you might want some sort of way to say "this patchset is version 2
   of that one", otherwise your statistics are going to be a bit weird
   if you think all the patches in v1 are "this was never accepted" and
   the patches in v2 have a time-to-commit starting from when v2 was
   posted rather than from when v1 was posted.

I believe this is possible, but I'm not sure how it is done. Guilherme?

You could mark the first patchset as 'superseded', I guess. With
the bulk-edit feature that's less effort than if it had to be done
one patch at a time.

Post by James Westby

Post by Peter Maydell
 * ability to add other people's patches (ie by non-Linaro
   people) to my "need to review this patch" list and to "will put into
   next pull request", etc. [I know this is a bit out of scope for
   linaro's metrics tracking, but I definitely don't want to have to
   track patches in more than one place if I can avoid it]

Yes, I think this is out of scope for the current project. I don't know
if we can do this very cheaply for you though, as I would agree that one
place to do all this would be good.

This case is currently sufficiently uncommon that I can probably live
with out of band tracking via wiki page if necessary. It would be nice
to know whether it's a '2 hours of coding' or '2 weeks of coding' level
feature though :-)

Post by James Westby

Post by Peter Maydell
requests (ie when I ask upstream to commit things)? I'm guessing
and doesn't need them again.

It sounds like there is no need. I'm also guessing that if you have all
patches reviewed first there isn't a long time between pull request and
pull?

It's a bit variable, since I only usually have to try the pull-request
approach if other people upstream have been too busy to review/apply
anyway. It doesn't kick in often.

-- PMM

[James Westby]

Post by Peter Maydell
Sure. It does mean that fixing deficiencies in the system is more
important than if it's just collecting patches and only needs to
be interacted with by a few people.

Agreed.

Post by Peter Maydell
It's a feature I'd really like to see, and upstream sounded
http://lists.ozlabs.org/pipermail/patchwork/2011-January/000348.html
so it would be great if we had the resources to do something here.

Thanks for the reference.

Post by Peter Maydell
Basically, some qemu patches I send just get committed, which
is the straightforward case. Some patches don't get any review
comments of any form, so they "time out" and I eventually try
to batch them up and resubmit them via a pull request. I want
to be able to distinguish the 'timed out' patches from the others.

For your own benefit so that you know to send the pull request?

It doesn't sound like we should track that for metrics?

Post by Peter Maydell
I'm not wedded to this particular set of states but it might
be nice for my purposes to have a little more flexibility.
I'll have a go with the existing set of states for a bit and
that ought to give me a better idea of whether I really need more.

That sounds good thanks.

Post by Peter Maydell
You could mark the first patchset as 'superseded', I guess. With
the bulk-edit feature that's less effort than if it had to be done
one patch at a time.

Yeah. It would be good to have "superseded by" for the metrics to do
"time from first submission to acceptance" and "average number of
re-works before acceptance" type things.

Post by Peter Maydell
This case is currently sufficiently uncommon that I can probably live
with out of band tracking via wiki page if necessary. It would be nice
to know whether it's a '2 hours of coding' or '2 weeks of coding' level
feature though :-)

I think that the difficulty here is that we are only tracking patches
sent to ***@linaro.org, not everything sent to the qemu list. I'm
not sure how you would add someone else's patches to the system in this
setup. You could send them to someone-elses-***@linaro.org, but then
we have to have a split in the database to know whether to include
patches in the metrics or not.

Given this I'm not sure how much effort it would be for us, but I'm
thinking that 2 weeks is a better estimate than 2 hours. Guilherme may
be able to make a better assessment than me.

Post by Peter Maydell
It's a bit variable, since I only usually have to try the pull-request
approach if other people upstream have been too busy to review/apply
anyway. It doesn't kick in often.

Right, it's a little more nuanced that I thought.

Thanks,

James

[Guilherme Salgado]

[...]

Post by Peter Maydell

Post by James Westby

Post by Peter Maydell
In that case I'd like it to be able to replace my
https://wiki.linaro.org/PeterMaydell/QemuPatchStatus
* ability to deal with whole patchsets as a single 'line item'
in patchwork, so you can move a 10-patch patchset from state to
state without it being a huge amount of drudgery

It can do this. It currently requires you to click to link the patches
from the set, but we've discussed ways to do this automatically in most
cases. I'm not sure if we have specific plans to improve this currently
though.

It's a feature I'd really like to see, and upstream sounded
http://lists.ozlabs.org/pipermail/patchwork/2011-January/000348.html
so it would be great if we had the resources to do something here.

Automatic grouping of patch series sounds like it wouldn't be a lot of
work, although we'd have to be careful with versions other than the
first one as their prefix ([v2,1/4], [2,1/4], etc) seem to vary.

Also, keeping the cover letter is trickier given the way patchwork
parses messages, but it's certainly doable as well.

Post by Peter Maydell

Post by James Westby

Post by Peter Maydell
* missing statuses for tracking patch progress; at the moment
'waiting for review'
'will put into next pull request'
'pull request sent, not committed'

It's not clear to me what these statuses mean? The patch is reviewed,
and once it is accepted it then goes in to a pull request and is
committed that way?

Basically, some qemu patches I send just get committed, which
is the straightforward case. Some patches don't get any review
comments of any form, so they "time out" and I eventually try
to batch them up and resubmit them via a pull request. I want
to be able to distinguish the 'timed out' patches from the others.
So "waiting for review" means "patch has gone on list and hasn't
had any comments"; "will put into next pull req" means "patch has
gone on list, had no comments but it's been a couple of weeks
so I consider it to have passed review by default"; "pull request
sent" means "I've sent upstream a pull request email but am waiting
for upstream to actually pull (or deny, as the case may be)".
"committed" means "actually got into upstream's git tree".
I'm not wedded to this particular set of states but it might
be nice for my purposes to have a little more flexibility.
I'll have a go with the existing set of states for a bit and
that ought to give me a better idea of whether I really need more.

We can very easily add new states (via the admin UI), but I don't think
we'll be able to automate transitions between all these states, so you'd
have to change the state manually before a patch is accepted/committed
-- at which point its state should be automatically updated.

Post by Peter Maydell

Post by James Westby

Post by Peter Maydell
* you ought to be able to change patch states from the top level
list by ticking ticky boxes; at the moment it looks like you have
to go into the individual patch page to do this. Combined with the
lack of any sensible handling of patchsets, this means that marking
a patchset as applied is just way too much work.

I believe that if you go to your page listing outstanding patches you
can bulk-edit. I agree that bulk editing is an important feature.

Looks like you can, yes. So not being able to do that from the
project page is just a UI inconsistency.

This is because Patchwork only allows project maintainers to change
state on patch lists, but I think we don't want that restriction, so
I'll get rid of it.

Post by Peter Maydell

Post by James Westby

Post by Peter Maydell
* you might want some sort of way to say "this patchset is version 2
of that one", otherwise your statistics are going to be a bit weird
if you think all the patches in v1 are "this was never accepted" and
the patches in v2 have a time-to-commit starting from when v2 was
posted rather than from when v1 was posted.

I believe this is possible, but I'm not sure how it is done. Guilherme?

You could mark the first patchset as 'superseded', I guess. With
the bulk-edit feature that's less effort than if it had to be done
one patch at a time.

We already ignore superseded patch series when generating the metrics,
so when there's a newer version of a patch/series, their submitters will
just have to mark the previous one as superseded.

The only problem is when a patch/series is marked superseded after the
turn of the month, as in that case the metrics will change after the end
of the month.

And that's similar to the way we track state changes (as we don't keep
track of the dates when they were changed), so every time a patch is
marked accepted, the metrics for the month when that patch was submitted
will change. This is good in the sense that we can get a better picture
of the submitted/accepted ratio, but if we take these metrics out and
publish them somewhere, then some data will be lost.

Post by Peter Maydell

Post by James Westby

Post by Peter Maydell
* ability to add other people's patches (ie by non-Linaro
people) to my "need to review this patch" list and to "will put into
next pull request", etc. [I know this is a bit out of scope for
linaro's metrics tracking, but I definitely don't want to have to
track patches in more than one place if I can avoid it]

Yes, I think this is out of scope for the current project. I don't know
if we can do this very cheaply for you though, as I would agree that one
place to do all this would be good.

This case is currently sufficiently uncommon that I can probably live
with out of band tracking via wiki page if necessary. It would be nice
to know whether it's a '2 hours of coding' or '2 weeks of coding' level
feature though :-)

I think it'd require a significant amount of discussion on how to
implement it first, which means it's definitely not on the '2h'
category.

Post by Peter Maydell

Post by James Westby

Post by Peter Maydell
requests (ie when I ask upstream to commit things)? I'm guessing
and doesn't need them again.

It sounds like there is no need. I'm also guessing that if you have all
patches reviewed first there isn't a long time between pull request and
pull?

It's a bit variable, since I only usually have to try the pull-request
approach if other people upstream have been too busy to review/apply
anyway. It doesn't kick in often.

[Michael Hope]

Post by James Westby
Hi,
This service is going to be used by management to get an idea of the
number of patches going to each project over time, the number of patches
submitted upstream by each team over time, the % of patches accepted
upstream, the average time from submission to acceptance for each
project, and other things like that.
For this to work well, and for your work to be accurately counted, you
are going to be expected to keep it up to date as you submit patches
upstream.
Now is the time to speak up if the interface doesn't have what you need
to do this. Obviously it would be better if it took care of everything
for you, but we don't think that everything can be automated. If you
know better then we would obviously like to hear about that too.

Hmm. We already do patch tracking in Linaro GCC to make sure that all
patches go upstream. It's a manual process as the GCC workflow itself
is very manual.

I don't want to manually update two places when a patch changes state.
How can we merge these systems?

-- Michael

[Guilherme Salgado]

Post by Michael Hope

Post by James Westby
Hi,
This service is going to be used by management to get an idea of the
number of patches going to each project over time, the number of patches
submitted upstream by each team over time, the % of patches accepted
upstream, the average time from submission to acceptance for each
project, and other things like that.
For this to work well, and for your work to be accurately counted, you
are going to be expected to keep it up to date as you submit patches
upstream.
Now is the time to speak up if the interface doesn't have what you need
to do this. Obviously it would be better if it took care of everything
for you, but we don't think that everything can be automated. If you
know better then we would obviously like to hear about that too.

Hmm. We already do patch tracking in Linaro GCC to make sure that all
patches go upstream. It's a manual process as the GCC workflow itself
is very manual.
I don't want to manually update two places when a patch changes state.
How can we merge these systems?

If you update the patch state in the new system, you get a list of
patches that can be filtered by state/submitter/etc, like:
http://ec2-184-73-78-92.compute-1.amazonaws.com/project/gcc-patches/list/

But I assume that's not everything you want so our long term goal should
be to have the features you need implemented directly into the system,
although in the meantime we could have them implemented as separate
tools, using our system as a data source via its xml-rpc API.

[James Westby]

Post by Michael Hope
Hmm. We already do patch tracking in Linaro GCC to make sure that all
patches go upstream. It's a manual process as the GCC workflow itself
is very manual.
I don't want to manually update two places when a patch changes state.
How can we merge these systems?

The original plan from UDS was to start from something like your system
and extend it to other teams. That changed when ***@linaro.org was
proposed in Dallas.

We can perhaps modify your script to create patches in patchwork rather
than bugs. I fear that may be a bit simplistic for your case though? Is
there a one-to-one correspondance between bugs and "patches" that are
being sent upstream?

In addition, what things do you need to track about each patch? It may
be that patchwork doesn't have all the fields needed to model your
workflow.

Thanks,

James

[Zach Pfeffer]

We're planning on bringing up Gerrit at UDS for tracking Android
upstreaming. Perhaps it could be used for tracking GCC patches?

Post by James Westby

Hmm.  We already do patch tracking in Linaro GCC to make sure that all
patches go upstream.  It's a manual process as the GCC workflow itself
is very manual.
I don't want to manually update two places when a patch changes state.
 How can we merge these systems?

The original plan from UDS was to start from something like your system
proposed in Dallas.
We can perhaps modify your script to create patches in patchwork rather
than bugs. I fear that may be a bit simplistic for your case though? Is
there a one-to-one correspondance between bugs and "patches" that are
being sent upstream?
In addition, what things do you need to track about each patch? It may
be that patchwork doesn't have all the fields needed to model your
workflow.
Thanks,
James
_______________________________________________
linaro-dev mailing list
http://lists.linaro.org/mailman/listinfo/linaro-dev

[Michael Hope]

Post by Zach Pfeffer
We're planning on bringing up Gerrit at UDS for tracking Android
upstreaming. Perhaps it could be used for tracking GCC patches?

Probably not. It seems to be tied to git and requires the Android
workflow. Here's our workflow:

Patches that are backported or unique to gcc-linaro are handled
through the Launchpad merge request system.

Patches on mainline are constrained by mainline's method which is:
* Send the patch to gcc-***@gcc.gnu.org
* Iterate on the mailing list, perhaps in new threads
* Get approval via reply to the latest post
* Commit into svn

The final commit is automatically emailed to the gcc-***@gcc.gnu.org list.

-- Michael