Difference between revisions of "Development Server"

From IridiaWiki
Jump to navigationJump to search
(do not mention rehan, direct people to lab responsabilities)
 
(8 intermediate revisions by 4 users not shown)
Line 3: Line 3:
   
 
* subversion repositories
  
* subversion repositories
  +
* git repositories
 
* 'trac' environments for projects (wiki, bugtracker etc)
  
* 'trac' environments for projects (wiki, bugtracker etc)
 
* webpages for projects
  
* webpages for projects
Line 9: Line 10:
 
can host this on the new server.
  
can host this on the new server.
   
−
== How to add a new subversion repository? ==
 +
== How to add a new subversion/git repository? ==
   
 
Currently, you have to contact the server administrator for that. See [[Lab responsibilities]] to find out who's the current administrator of the server and the repositories.
  
Currently, you have to contact the server administrator for that. See [[Lab responsibilities]] to find out who's the current administrator of the server and the repositories.
   
−
== How to access the subversion repositories? ==
 +
== How to access the subversion/git repositories? ==
   
 
You should get a username and a password from the repositories administrator. Instead of
  
You should get a username and a password from the repositories administrator. Instead of
Line 19: Line 20:
 
is the following:
  
is the following:
   
−
https://iridia-dev.ulb.ac.be/projects/%NAME%/svn
 +
https://iridia-dev.ulb.ac.be/projects/%NAME%/{svn|git}
   
 
For example, you can check out a project like this:
  
For example, you can check out a project like this:
Line 25: Line 26:
 
svn co https://%USERNAME%@iridia-dev.ulb.ac.be/projects/%REPNAME%/svn
  
svn co https://%USERNAME%@iridia-dev.ulb.ac.be/projects/%REPNAME%/svn
   
  +
== I want to delete/archive old repositories ==
  +
  +
Old repositories are stored in <tt>iridiadevserver@fallopius:~/archived_svn_repos</tt>
  +
  +
Ask the administrator to move it there (<tt>ssh fallopius-backup</tt> from iridia-dev as root).
   
 
[[Lab responsibilities|Contact the administrator]] if you have any problems/questions.
  
[[Lab responsibilities|Contact the administrator]] if you have any problems/questions.
  +
  +
== I want to share one subdirectory of my repository, but I don't want to give access to the rest of my repository ==
  +
  +
Normally, each repository is either all-or-nothing access (apart from read-write permissions), that is, if you give read access to someone, then they can read the whole repository. It is possible to [http://svnbook.red-bean.com/en/1.7/svn.serverconfig.pathbasedauthz.html setup per-directory access], but it is a bit more complicated and it requires a recent Apache and a recent SVN, which we don't have in iridia-dev.
  +
  +
First, ''do you really need this?'' See the link (http://svnbook.red-bean.com/en/1.7/svn.serverconfig.pathbasedauthz.html) why it is better to not rely on this setup and rather to either trust or not trust people. Everything in SVN is reversible, so mistakes can easily be undone.
  +
  +
If you really want to start a collaboration with someone and re-use some files in a repository but not give access to the whole repository, then please consider creating a new repository and copying the files. If you record revision info/url at the moment of the copy, for example, in the first commit to the new repository, you can always merge changes back and forth between repositories with <tt>svn merge</tt>. Most likely, you will need to merge just once (when the project is finished), and possibly never.
  +
  +
  +
== Quick SVN HowTo ==
  +
  +
[http://svnbook.red-bean.com/nightly/en/svn.intro.quickstart.html Subversion Quick Tutorial]: A very fast tutorial on Subversion (SVN).
  +
  +
== Quick Git HowTo ==
  +
  +
==== No password [Linux] ====
  +
  +
If you don't want to be prompted for your password at each operation you have to add in your home directory a .netrc file:
  +
  +
cat > ~/.netrc
  +
machine iridia-dev.ulb.ac.be
  +
login %YOUR_USERNAME%
  +
password %YOUR_PASSWORD%
  +
CTRL+D
  +
  +
and change permissions, so that you are the only one that can access it
  +
  +
chmod 600 ~/.netrc
  +
  +
Note that if you clone a repository specifying your username in the URL (e.g., https://%USERNAME%@iridia.dev...) your .netrc file will be ignored and you'll have to specify also the password in the URL. If you don't like the idea of your password being stored in clear check out this [http://manpages.ubuntu.com/manpages/precise/man1/git-credential-store.1.html Ubuntu page] about how to store your password in your keyring. If you still want the password to be asked every time and in a remote connection you want to type the password in the command line and not a pop-up window, set in your bash profile:
  +
  +
unset SSH_ASKPASS
  +
  +
==== No password [Os X] ====
  +
  +
On Os X, after asking your password the first time, it will be stored in your keychain:
  +
  +
curl http://github-media-downloads.s3.amazonaws.com/osx/git-credential-osxkeychain -o git-credential-osxkeychain
  +
sudo mkdir -p /usr/local/bin
  +
sudo mv git-credential-osxkeychain /usr/local/bin
  +
sudo chmod u+x /usr/local/bin/git-credential-osxkeychain
  +
git config --global credential.helper osxkeychain
  +
  +
==== Configuring Git ====
  +
  +
Then you have to add some information about yourself in the configuration:
  +
  +
git config --global user.name "Pinco Pallino"
  +
git config --global user.email "ppallino@iridia.ulb.ac.be"
  +
git config --global --add color.ui true
  +
  +
this will be default in git 2.0
  +
  +
git config --global push.default simple
  +
  +
this is already default, but you could change it with your preferred editor
  +
  +
git config --global core.editor 'vim'
  +
  +
to wrap lines in git diff
  +
  +
git config --global core.pager 'less -r'
  +
  +
==== Iridia certificate ====
  +
  +
'''Don't''' ignore the certificate verification globally
  +
  +
# git config --global http.sslverify "false"
  +
  +
don't do it. Instead, open the page of your git repository in Firefox (this has to be done just for the first repository), when you are prompted for adding an exception for the certificate, ask Firefox to show you the certificate and export it as a X.509 (PEM) without chain or other options and save it in a file like iridia-dev.ulb.ac.be.pem. Then add it as a sslCAInfo:
  +
  +
git config --global http.sslCAInfo /home/%YOUR_USERNAME%/iridia-dev.ulb.ac.be.pem
  +
  +
  +
Otherwise, go the ignorant way and disable the certificate locally for the single repository
  +
  +
git config http.sslVerify false
  +
  +
==== Other useful stuff ====
  +
  +
All the configuration options specified above go into a configuration file in your home directory, i.e., ~/.gitconfig. You can add further options manually directly in that file or with git config command line. For example I shamelessly copied these useful aliases [http://durdn.com/blog/2012/11/22/must-have-git-aliases-advanced-examples/ from here]:
  +
  +
<pre>
  +
[alias]
  +
# list commits in short form, with colors and branch/tag annotations
  +
ls = log --pretty=format:"%C(yellow)%h%Cred%d\\ %Creset%s%Cblue\\ [%cn]" --decorate
  +
  +
# list commits showing changed files is invoked
  +
ll = log --pretty=format:"%C(yellow)%h%Cred%d\\ %Creset%s%Cblue\\ [%cn]" --decorate --numstat
  +
  +
# list with no colors if you need to chain the out put with Unix pipes
  +
lnc = log --pretty=format:"%h\\ %s\\ [%cn]"
  +
  +
# list oneline commits showing dates (git lds -10)
  +
lds = log --pretty=format:"%C(yellow)%h\\ %ad%Cred%d\\ %Creset%s%Cblue\\ [%cn]" --decorate --date=short
  +
  +
# list oneline commits showing relative dates (git ld -10)
  +
ld = log --pretty=format:"%C(yellow)%h\\ %ad%Cred%d\\ %Creset%s%Cblue\\ [%cn]" --decorate --date=relative
  +
  +
# default look for short git log
  +
le = log --oneline --decorate
  +
  +
# all the commits related to a file, with the diff of the changes
  +
filelog = log -u
  +
fl = log -u
  +
  +
# show modified files in last commit
  +
dl = "!git ll -1"
  +
  +
# show a diff last commit
  +
dlc = diff --cached HEAD^
  +
  +
# content of commit (full diff) given revision (got ls -10; git dr 8cf2224)
  +
dr = "!f() { git diff "$1"^.."$1"; }; f"
  +
lc = "!f() { git ll "$1"^.."$1"; }; f"
  +
diffr = "!f() { git diff "$1"^.."$1"; }; f"
  +
  +
# find a file path in codebase (git f README.txt)
  +
f = "!git ls-files | grep -i"
  +
  +
# search/grep your entire codebase for a string
  +
grep = grep -Ii
  +
gr = grep -Ii
  +
  +
# grep from root folder
  +
gra = "!f() { A=$(pwd) && TOPLEVEL=$(git rev-parse --show-toplevel) && cd $TOPLEVEL && git grep --full-name -In $1 | xargs -I{} echo $TOPLEVEL/{} && cd $A; }; f"
  +
  +
# list all aliases
  +
la = "!git config -l | grep alias | cut -c 7-"
  +
  +
# show the last tag
  +
lasttag = describe --tags --abbrev=0
  +
lt = describe --tags --abbrev=0
  +
  +
# basic shortcuts
  +
cp = cherry-pick
  +
st = status -s
  +
cl = clone
  +
ci = commit
  +
co = checkout
  +
br = branch
  +
diff = diff --word-diff
  +
dc = diff --cached
  +
</pre>
  +
and I also added a global gitignore file:
  +
  +
git config --global core.excludesfile ~/.gitignore_global
  +
  +
containing some generic patterns for Linux and some specific ones for Os X:
  +
  +
cat > ~/.gitignore_global
  +
.DS_Store
  +
.DS_Store?
  +
._*
  +
.Spotlight-V100
  +
.Trashes
  +
Icon?
  +
ehthumbs.db
  +
Thumbs.db
  +
*~
  +
*.lock
  +
*.DS_Store
  +
*.swp
  +
*.out
  +
  +
==== Cloning your repository in a local directory ====
  +
  +
At this point with everything configured you should be ready to clone a repository
  +
  +
git clone https://iridia-dev.ulb.ac.be/projects/%REPNAME%/git %REPNAME%
  +
  +
if you are on Linux and you have configured your .netrc you should have no issues, otherwhise you have maybe to specify also your username:
  +
  +
git clone https://%YOUR_USERNAME%@iridia-dev.ulb.ac.be/projects/%REPNAME%/git %REPNAME%
  +
  +
and you will be prompted for a password. If you later decide to use a .netrc file, you have to remove your username from the reopository url:
  +
  +
cd %REPNAME%
  +
git remote set-url origin https://iridia-dev.ulb.ac.be/projects/%REPNAME%/git
  +
  +
On Os X, regardless of the username in the URL, if you installed the keychain helper, after typing the password the first time, you will not be prompted anymore.
  +
  +
==== Using Git for SVN users ====
  +
  +
You should definitely read this [http://learn.github.com/p/intro.html short introduction] and if you really want to understand what you are doing you should read this [http://git-scm.com/book slightly longer introduction]. ;)
  +
  +
Anyway if you want a blind correspondence of basic SVN commands here it is:
  +
  +
svn commit => git commit -a
  +
  +
this is actually a local commit, to push it to the repository you need to make:
  +
  +
git push origin --all
  +
  +
or simply
  +
  +
git push
  +
  +
for updating your local copy to the most up-to-date version on the repository
  +
  +
svn update => git pull
  +
  +
changing your mind about your last commit message before pushing it on the repository
  +
  +
git commit --amend -m 'Edited message.'
  +
  +
The other commands like git diff are very similar to the corresponding svn ones. Branching and tagging are of course different, but you know it already if you are planning to use git.
  +
  +
Anyway, this short list is here to help you if you need to quickly commit something and you are not planning to use Git for your project. On the contrary, if you want to understand why there is a '-a' after the git command, what is the staging area, how to cherry pick a commit on a branch, or how to change the history of the commits on the repository, read at least the [http://learn.github.com/p/intro.html short introduction].

Latest revision as of 11:18, 14 June 2018

IRIDIA has a development server called "iridia-dev". The server provides the following services:

  • subversion repositories
  • git repositories
  • 'trac' environments for projects (wiki, bugtracker etc)
  • webpages for projects

So if you have some software you would like to make open source, you can host this on the new server.

How to add a new subversion/git repository?

Currently, you have to contact the server administrator for that. See Lab responsibilities to find out who's the current administrator of the server and the repositories.

How to access the subversion/git repositories?

You should get a username and a password from the repositories administrator. Instead of accessing the server by ssh, you have to use https. The repository URL is the following: 

 https://iridia-dev.ulb.ac.be/projects/%NAME%/{svn%7Cgit}

For example, you can check out a project like this: 

 svn co https://%USERNAME%@iridia-dev.ulb.ac.be/projects/%REPNAME%/svn

I want to delete/archive old repositories

Old repositories are stored in iridiadevserver@fallopius:~/archived_svn_repos

Ask the administrator to move it there (ssh fallopius-backup from iridia-dev as root).

Contact the administrator if you have any problems/questions.

I want to share one subdirectory of my repository, but I don't want to give access to the rest of my repository

Normally, each repository is either all-or-nothing access (apart from read-write permissions), that is, if you give read access to someone, then they can read the whole repository. It is possible to setup per-directory access, but it is a bit more complicated and it requires a recent Apache and a recent SVN, which we don't have in iridia-dev.

First, do you really need this? See the link (http://svnbook.red-bean.com/en/1.7/svn.serverconfig.pathbasedauthz.html) why it is better to not rely on this setup and rather to either trust or not trust people. Everything in SVN is reversible, so mistakes can easily be undone.

If you really want to start a collaboration with someone and re-use some files in a repository but not give access to the whole repository, then please consider creating a new repository and copying the files. If you record revision info/url at the moment of the copy, for example, in the first commit to the new repository, you can always merge changes back and forth between repositories with svn merge. Most likely, you will need to merge just once (when the project is finished), and possibly never.


Quick SVN HowTo

Subversion Quick Tutorial: A very fast tutorial on Subversion (SVN).

Quick Git HowTo

No password [Linux]

If you don't want to be prompted for your password at each operation you have to add in your home directory a .netrc file: 

   cat > ~/.netrc
   machine iridia-dev.ulb.ac.be
   login %YOUR_USERNAME%
   password %YOUR_PASSWORD%
   CTRL+D

and change permissions, so that you are the only one that can access it 

   chmod 600 ~/.netrc

Note that if you clone a repository specifying your username in the URL (e.g., https://%USERNAME%@iridia.dev...) your .netrc file will be ignored and you'll have to specify also the password in the URL. If you don't like the idea of your password being stored in clear check out this Ubuntu page about how to store your password in your keyring. If you still want the password to be asked every time and in a remote connection you want to type the password in the command line and not a pop-up window, set in your bash profile: 

   unset SSH_ASKPASS

No password [Os X]

On Os X, after asking your password the first time, it will be stored in your keychain: 

   curl http://github-media-downloads.s3.amazonaws.com/osx/git-credential-osxkeychain -o git-credential-osxkeychain
   sudo mkdir -p /usr/local/bin
   sudo mv git-credential-osxkeychain /usr/local/bin
   sudo chmod u+x /usr/local/bin/git-credential-osxkeychain
   git config --global credential.helper osxkeychain

Configuring Git

Then you have to add some information about yourself in the configuration: 

   git config --global user.name "Pinco Pallino"
   git config --global user.email "ppallino@iridia.ulb.ac.be"
   git config --global --add color.ui true

this will be default in git 2.0 

   git config --global push.default simple

this is already default, but you could change it with your preferred editor 

   git config --global core.editor 'vim'

to wrap lines in git diff 

   git config --global core.pager 'less -r'

Iridia certificate

Don't ignore the certificate verification globally 

   # git config --global http.sslverify "false"

don't do it. Instead, open the page of your git repository in Firefox (this has to be done just for the first repository), when you are prompted for adding an exception for the certificate, ask Firefox to show you the certificate and export it as a X.509 (PEM) without chain or other options and save it in a file like iridia-dev.ulb.ac.be.pem. Then add it as a sslCAInfo: 

   git config --global http.sslCAInfo /home/%YOUR_USERNAME%/iridia-dev.ulb.ac.be.pem


Otherwise, go the ignorant way and disable the certificate locally for the single repository 

   git config http.sslVerify false

Other useful stuff

All the configuration options specified above go into a configuration file in your home directory, i.e., ~/.gitconfig. You can add further options manually directly in that file or with git config command line. For example I shamelessly copied these useful aliases from here: 

    [alias]
        # list commits in short form, with colors and branch/tag annotations
        ls = log --pretty=format:"%C(yellow)%h%Cred%d\\ %Creset%s%Cblue\\ [%cn]" --decorate

        # list commits showing changed files is invoked
        ll = log --pretty=format:"%C(yellow)%h%Cred%d\\ %Creset%s%Cblue\\ [%cn]" --decorate --numstat

        # list with no colors if you need to chain the out put with Unix pipes
        lnc = log --pretty=format:"%h\\ %s\\ [%cn]"

        # list oneline commits showing dates (git lds -10)
        lds = log --pretty=format:"%C(yellow)%h\\ %ad%Cred%d\\ %Creset%s%Cblue\\ [%cn]" --decorate --date=short

        # list oneline commits showing relative dates (git ld -10)
        ld = log --pretty=format:"%C(yellow)%h\\ %ad%Cred%d\\ %Creset%s%Cblue\\ [%cn]" --decorate --date=relative

        # default look for short git log
        le = log --oneline --decorate

        # all the commits related to a file, with the diff of the changes
        filelog = log -u
        fl = log -u

        # show modified files in last commit
        dl = "!git ll -1"

        # show a diff last commit
        dlc = diff --cached HEAD^

        # content of commit (full diff) given revision (got ls -10; git dr 8cf2224)
        dr  = "!f() { git diff "$1"^.."$1"; }; f"
        lc  = "!f() { git ll "$1"^.."$1"; }; f"
        diffr  = "!f() { git diff "$1"^.."$1"; }; f"

        # find a file path in codebase (git f README.txt)
        f = "!git ls-files | grep -i"

        # search/grep your entire codebase for a string
        grep = grep -Ii
        gr = grep -Ii

        # grep from root folder
        gra = "!f() { A=$(pwd) && TOPLEVEL=$(git rev-parse --show-toplevel) && cd $TOPLEVEL && git grep --full-name -In $1 | xargs -I{} echo $TOPLEVEL/{} && cd $A; }; f"

        # list all aliases
        la = "!git config -l | grep alias | cut -c 7-"

        # show the last tag
        lasttag = describe --tags --abbrev=0
        lt = describe --tags --abbrev=0

        # basic shortcuts
        cp = cherry-pick
        st = status -s
        cl = clone
        ci = commit
        co = checkout
        br = branch
        diff = diff --word-diff
        dc = diff --cached

and I also added a global gitignore file: 

   git config --global core.excludesfile ~/.gitignore_global

containing some generic patterns for Linux and some specific ones for Os X: 

   cat > ~/.gitignore_global
   .DS_Store
   .DS_Store?
   ._*
   .Spotlight-V100
   .Trashes
   Icon?
   ehthumbs.db
   Thumbs.db
   *~
   *.lock
   *.DS_Store
   *.swp
   *.out

Cloning your repository in a local directory

At this point with everything configured you should be ready to clone a repository 

   git clone https://iridia-dev.ulb.ac.be/projects/%REPNAME%/git %REPNAME%

if you are on Linux and you have configured your .netrc you should have no issues, otherwhise you have maybe to specify also your username: 

   git clone https://%YOUR_USERNAME%@iridia-dev.ulb.ac.be/projects/%REPNAME%/git %REPNAME%

and you will be prompted for a password. If you later decide to use a .netrc file, you have to remove your username from the reopository url: 

   cd %REPNAME%
   git remote set-url origin https://iridia-dev.ulb.ac.be/projects/%REPNAME%/git

On Os X, regardless of the username in the URL, if you installed the keychain helper, after typing the password the first time, you will not be prompted anymore.

Using Git for SVN users

You should definitely read this short introduction and if you really want to understand what you are doing you should read this slightly longer introduction. ;)

Anyway if you want a blind correspondence of basic SVN commands here it is: 

   svn commit => git commit -a

this is actually a local commit, to push it to the repository you need to make: 

   git push origin --all

or simply 

   git push

for updating your local copy to the most up-to-date version on the repository 

   svn update => git pull

changing your mind about your last commit message before pushing it on the repository 

   git commit --amend -m 'Edited message.'

The other commands like git diff are very similar to the corresponding svn ones. Branching and tagging are of course different, but you know it already if you are planning to use git.

Anyway, this short list is here to help you if you need to quickly commit something and you are not planning to use Git for your project. On the contrary, if you want to understand why there is a '-a' after the git command, what is the staging area, how to cherry pick a commit on a branch, or how to change the history of the commits on the repository, read at least the short introduction.

Retrieved from "https://iridia.ulb.ac.be/w/index.php?title=Development_Server&oldid=8047"