Hi Daniel, Andrei, Jan, etc.
I have been trying to learn Git so that I can maybe help with the documentation on the project like I said once I would. Along the way, I have become curious how the project is managed as it relates to Git.
I have read over Jan's impromptu Git tutorial that he posted on the Kamailio-devel list some time ago to try to help out people switching from SVN, but still have some questions.
I am curious, what methodologies and tools do you use for hosting the central Git repository for sip-router? For instance, what scripts and tools are used to generate e-mails about submitted patches to the sr-dev list -- are they standard ones that come with Git, or something handcrafted or third-party? Also, how do you handle automated pushing and pulling of various developers' branches? Does the system only pull your personal branch, so that any topic branches you make are purely your own and when you want them put into the mainline code, you merge them into your main branch so they get pulled upstream? What does the workflow/process like?
Thank you!
-- Alex
On Sep 29, 2010 at 19:58, Alex Balashov abalashov@evaristesys.com wrote:
Hi Daniel, Andrei, Jan, etc.
I have been trying to learn Git so that I can maybe help with the documentation on the project like I said once I would. Along the way, I have become curious how the project is managed as it relates to Git.
I have read over Jan's impromptu Git tutorial that he posted on the Kamailio-devel list some time ago to try to help out people switching from SVN, but still have some questions.
I am curious, what methodologies and tools do you use for hosting the central Git repository for sip-router? For instance, what scripts and tools are used to generate e-mails about submitted patches to the sr-dev list -- are they standard ones that come with Git, or something handcrafted or third-party? Also, how do you handle automated pushing and pulling of various developers' branches? Does the system only pull your personal branch, so that any topic branches you make are purely your own and when you want them put into the mainline code, you merge them into your main branch so they get pulled upstream? What does the workflow/process like?
Repository:
Each developer has an account on the git machine (with a restricted git-shell). We have a modified update hook that takes care of permissions and calls the mail script. The update script is a modified version (I don't remember how "heavy") of http://www.kernel.org/pub/software/scm/git/docs/howto/update-hook-example.tx... (or at least of the version that was there 2 years ago).
For the commit permission the update hook looks into another file, in which we have a list of:
branch_pattern user_pattern_list
(both of them are regular expressions)
If the branch_pattern and user_pattern match for a specific commit, the commit is allowed. Besides that any commit on a "username" branch (of the form $user/.*) is allowed as long as the username of the commiter is the same as $user (this can be turned off using a config option).
The relevant parts of the branch permission file for sip_router are:
# everybody can do any change in tmp/* branches +refs/heads/tmp/.* .*
# everybody can do any change in kamailio/* branches # - used to release kamailio stable versions # - naming pattern: kamailio/major-version: kamailio/3.0 refs/heads/kamailio/.* .*
# everybody can commit on the master branch refs/heads/master$ .* refs/heads/sr_3.0$ .* refs/heads/kamailio_3.0$ .*
# everybody can make any tag refs/tags/.* .*
[besides that there are some restriction for old branches inherited from ser and for the ser sync branches, which are not used anymore]
The mail script is again a modified version of some perl script. I do not remember were we took it from. Here is its header:
# V0.4 (mails for never seen before commits) # # Tool to send git commit notifications # # Copyright 2005 Alexandre Julliard # # Updated by Jan Janak jan@iptel.org # Updated by Andrei Pelinescu-Onciul andrei@iptel.org
We also have an email_map file, mapping local usernames on the machine hosting the repo to the email addresses that you will see in the From: line of the commit mail. If no match is found (somebody forgot to update email_map), the from line will use admin@...).
Workflow:
There is no automatic pulling or pushing. The developer chooses the branch on which he wants to push his changes. As long as he has write permissions on that branch it will work. While a more restricted workflow would be theoretically better (e.g. each developer can push only on his branches or some tmp/ branches and then send pull/merge requests to some maintainer which would merge the developer branch into master or stable), most people are used with the CVS/SVN one branch-for-all model.
In general I try to use separate branches for bigger features (requiring several commits). It's easier for me (because I don't have to update the code everyday, I can work on the old version and merge it when ready and in the meantime I can switch between several branches with work-in-progress), it's easier to completely revert if needed and the history looks nicer. If I have only small fixes or small features I commit directly to the master branch. I also have several local branches for testing various smaller features, that I constantly rebase and push to the public repository only when ready.
Andrei P.S.: if you want to see the scripts I can send them to you
Andrei,
Thank you for your reply -- it is very informative! A few follow-up questions below:
On 09/30/2010 07:06 AM, Andrei Pelinescu-Onciul wrote:
For the commit permission the update hook looks into another file, in which we have a list of:
branch_pattern user_pattern_list
(both of them are regular expressions)
If the branch_pattern and user_pattern match for a specific commit, the commit is allowed. Besides that any commit on a "username" branch (of the form $user/.*) is allowed as long as the username of the commiter is the same as $user (this can be turned off using a config option).
Do you know if this restriction mechanism also part of a standard suite of scripts that are associated with git in one way or another, or something completely custom that was written specifically to manage permissions centrally on your side?
The mail script is again a modified version of some perl script. I do not remember were we took it from. Here is its header:
# V0.4 (mails for never seen before commits) # # Tool to send git commit notifications # # Copyright 2005 Alexandre Julliard # # Updated by Jan Janakjan@iptel.org # Updated by Andrei Pelinescu-Onciulandrei@iptel.org
Oh, okay. Yeah, I am familiar with this script. I was just wondering which one you were using, even if you have hacked it.
There is no automatic pulling or pushing. The developer chooses the branch on which he wants to push his changes. As long as he has write permissions on that branch it will work. While a more restricted workflow would be theoretically better (e.g. each developer can push only on his branches or some tmp/ branches and then send pull/merge requests to some maintainer which would merge the developer branch into master or stable), most people are used with the CVS/SVN one branch-for-all model.
So, all contributions to the central repository are ultimately done by pushing, never pulling?
Do all validated developers have write access to the master branches? Or do 'core', everyday developers like you have write on the master branch, while more distant, occasional contributors only have write to branches named after themselves, and core developers review/merge their changes?
Do you do any "cherry-picking" from time to time from smaller or less consistent contributors? If so, how do you do it? The only process with which I am familiar to do this is to 'git fetch' (not pull) someone else's changes, 'git cherry-pick' them out of the list of commits that appear in the range HEAD..FETCH_HEAD (or master..eval_repo if you 'git fetch'ed those changes straight into a new branch called 'eval_repo'), then blow the rest of the FETCH_HEAD/eval_repo differences away (git checkout master; git merge eval_repo -s ours). Do you do something like this?
In general I try to use separate branches for bigger features (requiring several commits). It's easier for me (because I don't have to update the code everyday, I can work on the old version and merge it when ready and in the meantime I can switch between several branches with work-in-progress), it's easier to completely revert if needed and the history looks nicer.
How do you handle the integration of those branches into master? Do you keep them local only while you are developing, and then, when you are ready to push them into the master on the central repository, do something like:
1. git checkout master 2. git pull [to update the local copy of the master branch from server] 3. git checkout feature_branch 4. git rebase master [solve any conflicts between master and feature_branch here, better to have the conflicts in the branch than in the master] 5. git checkout master 6. git merge feature_branch 7. git push
Or do you keep even your experimental / large feature branches pushed to the central repository commonly so that people can see those changes and look over them and cherry pick things out of them perhaps?
Also, what are your commit policies like? As you know, Git allows much more granular management of commits so that they can be organised to contain hunks that reflect related changes conceptually, rather than just unavoidably being a log of files changed over a certain period of time (like SVN or CVS). Do you insist that developers make very clean commits, stepping through each patch hunk with 'git add -p' and classifying it, or is this a matter of personal custom, habit and taste? Do you go back, as a maintainer, and compact a lot of small commits into big ones by rebasing them often?
In general, it sounds to me like you are mainly using Git in rather centralised, SVN/CVS-like ways, but in ways that take advantage of Git's cheap branching more effectively. Other than being able to branch easily -- which you don't really even do a lot of -- what do you perceive to be the main advantage of using Git?
Thank you for your insights!
-- Alex
On Sep 30, 2010 at 09:36, Alex Balashov abalashov@evaristesys.com wrote:
Andrei,
Thank you for your reply -- it is very informative! A few follow-up questions below:
On 09/30/2010 07:06 AM, Andrei Pelinescu-Onciul wrote:
For the commit permission the update hook looks into another file, in which we have a list of:
branch_pattern user_pattern_list
(both of them are regular expressions)
If the branch_pattern and user_pattern match for a specific commit, the commit is allowed. Besides that any commit on a "username" branch (of the form $user/.*) is allowed as long as the username of the commiter is the same as $user (this can be turned off using a config option).
Do you know if this restriction mechanism also part of a standard suite of scripts that are associated with git in one way or another, or something completely custom that was written specifically to manage permissions centrally on your side?
The update hook is derived from a standard example (which at least at that time was distributed with git). We added the "username" branch support and config options for the default policy.
The mail script is again a modified version of some perl script. I do not remember were we took it from. Here is its header:
# V0.4 (mails for never seen before commits) # # Tool to send git commit notifications # # Copyright 2005 Alexandre Julliard # # Updated by Jan Janakjan@iptel.org # Updated by Andrei Pelinescu-Onciulandrei@iptel.org
Oh, okay. Yeah, I am familiar with this script. I was just wondering which one you were using, even if you have hacked it.
There is no automatic pulling or pushing. The developer chooses the branch on which he wants to push his changes. As long as he has write permissions on that branch it will work. While a more restricted workflow would be theoretically better (e.g. each developer can push only on his branches or some tmp/ branches and then send pull/merge requests to some maintainer which would merge the developer branch into master or stable), most people are used with the CVS/SVN one branch-for-all model.
So, all contributions to the central repository are ultimately done by pushing, never pulling?
Yes.
Do all validated developers have write access to the master branches? Or do 'core', everyday developers like you have write on the master branch, while more distant, occasional contributors only have write to branches named after themselves, and core developers review/merge their changes?
All developers have access to the master and release branches. The username branches are writeable only by their creators. Apart from that we have a few old ser and ser integration branches on which I am the only one allowed to commit and others that are read-only for everybody. New branches can be created only under $username or tmp/ (this helps avoid typos in pushes, which otherwise would require constant miss-named branches deletion).
IMHO at least the release branches permissions should be restricted (only a group of release managers should be allowed to modify them), but OTOH hand so far it worked perfectly.
Do you do any "cherry-picking" from time to time from smaller or less consistent contributors?
We cherry-pick fixes from master into the release branches. So far we did not need to cherry-pick from contributors branches (so far we always ended up in merging the whole branch).
If so, how do you do it? The only process with which I am familiar to do this is to 'git fetch' (not pull) someone else's changes, 'git cherry-pick' them out of the list of commits that appear in the range HEAD..FETCH_HEAD (or master..eval_repo if you 'git fetch'ed those changes straight into a new branch called 'eval_repo'), then blow the rest of the FETCH_HEAD/eval_repo differences away (git checkout master; git merge eval_repo -s ours). Do you do something like this?
No, so far I don't no anybody that uses its own public repo. In the best case (when not pushing directly in master) everybody pushes his branches in the main repo and we merge from there. In general the maintainer of the piece of code affected will do the merge or ack it (so far there was no need of enforcing this).
In general I try to use separate branches for bigger features (requiring several commits). It's easier for me (because I don't have to update the code everyday, I can work on the old version and merge it when ready and in the meantime I can switch between several branches with work-in-progress), it's easier to completely revert if needed and the history looks nicer.
How do you handle the integration of those branches into master? Do you keep them local only while you are developing, and then, when you are ready to push them into the master on the central repository, do something like:
I always push to the repository the branches containing bigger features (the one I was speaking above). This allows other developers to see what I'm doing and acts also as a backup :-). For example I did all the tls async stuff in andrei/tcp_tls_changes. After some testing, when I was happy with it I merged it into master. I used: git checkout master; git merge --log --no-ff origin/andrei/tcp_tls_changes
I use something similar to what you describe below for local branches (smaller features, only a few commits that I push directly into master).
- git checkout master
- git pull [to update the local copy of the master branch from server]
I use either git pull --rebase origin master or git fetch; git rebase origin/master (in case I have some un-pushed stuff in my local master branch).
- git checkout feature_branch
- git rebase master [solve any conflicts between master and
feature_branch here, better to have the conflicts in the branch than in the master]
- git checkout master
- git merge feature_branch
- git push
Instead of the last 3 steps, I use directly: git push origin HEAD:master (less things to write)
Or do you keep even your experimental / large feature branches pushed to the central repository commonly so that people can see those changes and look over them and cherry pick things out of them perhaps?
Yes (although so far nobody was cherry-picking, but somebody might test stuff on the branches and can comment on the commits).
Also, what are your commit policies like? As you know, Git allows much more granular management of commits so that they can be organised to contain hunks that reflect related changes conceptually, rather than just unavoidably being a log of files changed over a certain period of time (like SVN or CVS). Do you insist that developers make very clean commits, stepping through each patch hunk with 'git add -p' and classifying it, or is this a matter of personal custom, habit and taste? Do you go back, as a maintainer, and compact a lot of small commits into big ones by rebasing them often?
I do try to group commits and I use git add -i _very_ _often_ to separate multiple changes to the same file that don't logically belong together. I also try to commit often and make smaller commits (easier to debug, bisect and if needed revert). We do recommend that the commits should be small and contain only related stuff, but so far we haven't enforced it.
We cannot rebase commits on public branches (that would break git pull for people tracking the branches).
In general, it sounds to me like you are mainly using Git in rather centralised, SVN/CVS-like ways, but in ways that take advantage of Git's cheap branching more effectively. Other than being able to branch easily -- which you don't really even do a lot of -- what do you perceive to be the main advantage of using Git?
Yes, we do use git in a centralised way. The reason for this is that this is what most developers are used to. In time we might gradually change to a more distributed model.
It's very hard for me to say what is the main advantage of using git. Now I'm so used with it, that it's like asking what is the main advantage of breathing air :-) I guess being distributed is the most useful feature. Apart from that I find it extremely developer/maintainer friendly. I couldn't leave now without rebase or add -i. Besides that there are all the little things like git describe, git branch --contains, git format-patch, git-am, git send-email, git grep, git log branch1 ^branch2. It's also very fast and very flexible (it doesn't enforce a specific model).
Andrei
-
Alex Balashov -
Andrei Pelinescu-Onciul