IridiaWiki - User contributions [en]

Talk:Using the IRIDIA Cluster

2006-04-25T08:10:51Z

Hlabella: /* Max's comments */

= Discussion on who is supposed to use the cluster, and abuse =

* Outcome of admin meeting of Thursday 20th April 2006

:: Who can use the cluster

::: Only people directly associated with IRIDIA (students, post docs, etc.) can use the cluster. Researchers who left should arrange with their contact/collaborator/supervisor here, to use his/her login in case they need to use the cluster.

:: The cluster-policy

::: Max explained the cluster policy and a mem limit might be introduced.

* Elio's thoughts :

<pre>
Dear all,

please, take my thoughts in a constructive way.

I read on the IRIDIA Wiki page the following statement, which I take as
the official position of the Lab:

"Only people directly associated with IRIDIA (students, post docs, etc.)
can use the cluster. Researchers who left should arrange with their
contact/collaborator/supervisor here, to use his/her login in case they
need to use the cluster"

Taken from:
http://iridia.ulb.ac.be/wiki/index.php/Administration_weekly_meetings#Thursday_20th_April_2006

With respect to the above statement I would like to add some comments.
In my opinion, there are at least two reasons which may suggest to
undertake a different direction to free the cluster from over usage. I
believe that, the idea of revealing to "trusted" people the
login/password of IRIDIA members is not the most efficient one because:

1) The recent history teaches us that even trusted people can cause
problems. We experienced such a case in a not recent past when an IRIDIA
member created some trouble to Halva which was maintening the cluster
at that time. I don't want to stigmatize the case. I DO believe that
that action was more the result of an unexperienced person rather then
the result of a real intetion to cause demages. However, it teaches us
that even trusted people can cause problems. This is obviously always
the case, but if each one as its own login/password it is easier to spot
the responsibles, I guess, and sorry if I am wrong on this.

2) If using someone else login/password becomes the official practice
for the IRIDIA cluster, as I read in the above statement, then even the
trusted people can recursively apply the same practice and in a while we
might have more jobs runnig then what we have at the moment. Do we have
tools to prevent this from happening?

Thanks a lot for your attention.
Cheers,
Elio
</pre>

:: Alex :
:: I agree with what Elio says. A login '''must''' be personal, and associated to one person only. If you want to prevent somebody from accessing the cluster, just disable the account. If we trust that person then it is fine, and we can let her access the cluster.

:: In general of course, there should be a kind of period during which an account is still usable. At least to retrieve files, to launche the five last replications that you always delayed and so on. Accounts should be simply deleted if that period is over and the owner of the account is no more doing something with IRIDIA. If someone is collaborating with a member of IRIDIA I think it is ok to let him have an account.

:: If we disagree with way people use the cluster, then either :
::: we find ways to define an efficient policy of cluster usage.
::: or we point out the problems and discuss them. It is anyway a matter of fairness.

:: I think that the cluster of IRIDIA is very good and unless everybody wants to make simulations at the same time, it is enough for all of us. Users should design carefully their source code to make it fast. Users should also avoid using the cluster for debugging purpose. You should not submit a set of simulations just to see if your code works. That you can do on your own computer and this way you respect the common resources.

:: As it is a common resource, we should also understand that it may not be immediately accessible. So when you forecast running a simulation you should know in advance that you may have to wait 1 day to have your jobs running.

== Halva's comments ==

I do believe that everyone should have its own login. Logins are no restricted resources, thus we can have as much as want of them. If you fear that users might misuse the cluster (as the example Elio was referring to), there are ways to tighten the kernel security and the resource usage. If anybody wants to look into the matter, and if she/he HAS time to do it, she/he is welcome! :)

I think also that the current policy is pretty fair. A job gets a lower priority if the user is already running other jobs on the cluster. Thus people with few jobs have higher chances to see their jobs be promptly run. Additionaly, the current policy limits the job duration, forcing the user to think twice before submitting anything.

Is it not enough? One might think to hack the queueing system scheduler to give even lower priority to those how have used the cluster the most in the last period. Or we might consider to buy a professional queueing system, which might give finer control on its mechanism.

Is it still not enough? Then go on <tt>polyphemus.ulb.ac.be</tt>, issue <tt>qacct -o</tt> to see who has heavily used the cluster (<tt>qacct -o -d 30</tt> to see the statistics of the last 30 days) and ''punish'' her/him :)

== Max's comments ==

I believe that, under normal circumnstances, only people affiliated to IRIDIA should have accounts on the cluster. Of course, special agreements to open external accounts can be made, but this is more a political decision of the "bosses" :)
There was a plan to "harden" the security of IRIDIA network putting every machine inside an internal network with no direct connection from the outside. I still support that view, the less "access points" we leave open from the outside world, the better, therefore I think that if I collaborate with somebody external to IRIDIA, I should be responsible for the usage of IRIDIA resources, and I should launch the experiments on the cluster with my account. If the sources are provided by the external, or if the results will be analyzed by the external, nobody cares.
* let's exercise a bit in a conspirancy theory: you closed the cluster in a local network; someone gives you some code to run the experiments with your account; the code contains malicious lines that jopardize the cluster => closing the cluster in a local network does not help, and you become the bad guy who hacked the cluster :)--[[User:Hlabella|haiax]] 10:07, 25 April 2006 (CEST)

User:Hlabella

2006-04-25T08:09:56Z

Hlabella:

'''Name:''' Thomas Halva Labella

'''Home Page:''' [http://iridia.ulb.ac.be/~hlabella http://iridia.ulb.ac.be/~hlabella]

Talk:Using the IRIDIA Cluster

2006-04-25T08:07:12Z

Hlabella: /* Max's comments */

= Discussion on who is supposed to use the cluster, and abuse =

* Outcome of admin meeting of Thursday 20th April 2006

:: Who can use the cluster

::: Only people directly associated with IRIDIA (students, post docs, etc.) can use the cluster. Researchers who left should arrange with their contact/collaborator/supervisor here, to use his/her login in case they need to use the cluster.

:: The cluster-policy

::: Max explained the cluster policy and a mem limit might be introduced.

* Elio's thoughts :

<pre>
Dear all,

please, take my thoughts in a constructive way.

I read on the IRIDIA Wiki page the following statement, which I take as
the official position of the Lab:

"Only people directly associated with IRIDIA (students, post docs, etc.)
can use the cluster. Researchers who left should arrange with their
contact/collaborator/supervisor here, to use his/her login in case they
need to use the cluster"

Taken from:
http://iridia.ulb.ac.be/wiki/index.php/Administration_weekly_meetings#Thursday_20th_April_2006

With respect to the above statement I would like to add some comments.
In my opinion, there are at least two reasons which may suggest to
undertake a different direction to free the cluster from over usage. I
believe that, the idea of revealing to "trusted" people the
login/password of IRIDIA members is not the most efficient one because:

1) The recent history teaches us that even trusted people can cause
problems. We experienced such a case in a not recent past when an IRIDIA
member created some trouble to Halva which was maintening the cluster
at that time. I don't want to stigmatize the case. I DO believe that
that action was more the result of an unexperienced person rather then
the result of a real intetion to cause demages. However, it teaches us
that even trusted people can cause problems. This is obviously always
the case, but if each one as its own login/password it is easier to spot
the responsibles, I guess, and sorry if I am wrong on this.

2) If using someone else login/password becomes the official practice
for the IRIDIA cluster, as I read in the above statement, then even the
trusted people can recursively apply the same practice and in a while we
might have more jobs runnig then what we have at the moment. Do we have
tools to prevent this from happening?

Thanks a lot for your attention.
Cheers,
Elio
</pre>

:: Alex :
:: I agree with what Elio says. A login '''must''' be personal, and associated to one person only. If you want to prevent somebody from accessing the cluster, just disable the account. If we trust that person then it is fine, and we can let her access the cluster.

:: In general of course, there should be a kind of period during which an account is still usable. At least to retrieve files, to launche the five last replications that you always delayed and so on. Accounts should be simply deleted if that period is over and the owner of the account is no more doing something with IRIDIA. If someone is collaborating with a member of IRIDIA I think it is ok to let him have an account.

:: If we disagree with way people use the cluster, then either :
::: we find ways to define an efficient policy of cluster usage.
::: or we point out the problems and discuss them. It is anyway a matter of fairness.

:: I think that the cluster of IRIDIA is very good and unless everybody wants to make simulations at the same time, it is enough for all of us. Users should design carefully their source code to make it fast. Users should also avoid using the cluster for debugging purpose. You should not submit a set of simulations just to see if your code works. That you can do on your own computer and this way you respect the common resources.

:: As it is a common resource, we should also understand that it may not be immediately accessible. So when you forecast running a simulation you should know in advance that you may have to wait 1 day to have your jobs running.

== Halva's comments ==

I do believe that everyone should have its own login. Logins are no restricted resources, thus we can have as much as want of them. If you fear that users might misuse the cluster (as the example Elio was referring to), there are ways to tighten the kernel security and the resource usage. If anybody wants to look into the matter, and if she/he HAS time to do it, she/he is welcome! :)

I think also that the current policy is pretty fair. A job gets a lower priority if the user is already running other jobs on the cluster. Thus people with few jobs have higher chances to see their jobs be promptly run. Additionaly, the current policy limits the job duration, forcing the user to think twice before submitting anything.

Is it not enough? One might think to hack the queueing system scheduler to give even lower priority to those how have used the cluster the most in the last period. Or we might consider to buy a professional queueing system, which might give finer control on its mechanism.

Is it still not enough? Then go on <tt>polyphemus.ulb.ac.be</tt>, issue <tt>qacct -o</tt> to see who has heavily used the cluster (<tt>qacct -o -d 30</tt> to see the statistics of the last 30 days) and ''punish'' her/him :)

== Max's comments ==

I believe that, under normal circumnstances, only people affiliated to IRIDIA should have accounts on the cluster. Of course, special agreements to open external accounts can be made, but this is more a political decision of the "bosses" :)
There was a plan to "harden" the security of IRIDIA network putting every machine inside an internal network with no direct connection from the outside. I still support that view, the less "access points" we leave open from the outside world, the better, therefore I think that if I collaborate with somebody external to IRIDIA, I should be responsible for the usage of IRIDIA resources, and I should launch the experiments on the cluster with my account. If the sources are provided by the external, or if the results will be analyzed by the external, nobody cares.
* let's exercise a bit in a conspirancy theory: you closed the cluster in a local network; someone gives you some code to run the experiments with your account; the code contains malicious lines that jopardize the cluster => closing the cluster in a local network does not help, and you become the bad guy who has hacked the cluster :)--[[User:Hlabella|haiax]] 10:07, 25 April 2006 (CEST)

2005-12-14T15:30:56Z

Hlabella: /* What are we gonna drink? */

=!!!!!!! PARTY !!!!!!!=

===What are we gonna eat?===
#Cotechino & Lenticchie (Lenticchie prepared by Vito, cotechino prepared by anyone). Ingredients:
#* lenticchie (Vito can buy in south italy ;-) ), celery, carots, onionsm, garlic, olive oil
#* cotechino
#Ciciri e tria (typical salentu dish prepared by Vito... someone already tried, but as Vito was drunk that time, he used a bit too much pepper... He promises he won't use pepper this time!). Ingredients can be bought by Vito in south Italy...

===What are we gonna drink?===

# Negroni
#* Simple to prepare. We need: Campari, Gin, Red Martini.
#** The real problem is "how much"...

===What are we gonna buy?===

New year

2005-12-14T15:30:31Z

Hlabella: /* What are we gonna drink? */

=!!!!!!! PARTY !!!!!!!=

===What are we gonna eat?===
#Cotechino & Lenticchie (Lenticchie prepared by Vito, cotechino prepared by anyone). Ingredients:
#* lenticchie (Vito can buy in south italy ;-) ), celery, carots, onionsm, garlic, olive oil
#* cotechino
#Ciciri e tria (typical salentu dish prepared by Vito... someone already tried, but as Vito was drunk that time, he used a bit too much pepper... He promises he won't use pepper this time!). Ingredients can be bought by Vito in south Italy...

===What are we gonna drink?===

# Negroni
** Simple to prepare. We need: Campari, Gin, Red Martini.
*** The real problem is "how much"...

===What are we gonna buy?===

New year

2005-12-14T15:29:43Z

Hlabella: /* What are we gonna drink? */

=!!!!!!! PARTY !!!!!!!=

===What are we gonna eat?===
#Cotechino & Lenticchie (Lenticchie prepared by Vito, cotechino prepared by anyone). Ingredients:
#* lenticchie (Vito can buy in south italy ;-) ), celery, carots, onionsm, garlic, olive oil
#* cotechino
#Ciciri e tria (typical salentu dish prepared by Vito... someone already tried, but as Vito was drunk that time, he used a bit too much pepper... He promises he won't use pepper this time!). Ingredients can be bought by Vito in south Italy...

===What are we gonna drink?===

* Negroni
** Simple to prepare. We need: Campari, Gin, Red Martini.
*** The real problem is "how much"...

===What are we gonna buy?===

History Halva

2005-12-05T11:28:44Z

Hlabella: /* History (past work) */

= History (past work) =

My past research focused on methods for task allocation and division of labour in a group of robots.

At the beginning, I implemented a mechanism for division of labour in a group of robots. The particular algorithm was inspired by ants' foraging behaviour. I studied its characteristics and features in a set of experiments with real robots and in simulation.

The knowledge obtained by this work was used then to compare different solutions to the division-of-labour problem. Different algorithms from the literature were adapted for a prey retrieval task and tested in common and fair environment. The experiments allow us to draw reliable conclusions about which is the most efficient or the most performing algorithm.

= Conferences attended =

{| border=1 cellspacing=0 cellpadding=2
! Name !! Location !! Dates
|-
| From Worker to Colony: Understanding the Organisation of Insect Societies || Cambridge, UK || December 7-8, 2001
|-
| IEEE International Conference on System, Man and Cybernetics (SMC2002) || Hammamet, Tunisia || October 6-9, 2002
|-
| Third International Workshop on Ants Algorithm (ANTS 2002) || Brussels, Belgium || September 12-14, 2002
|-
| First International Workshop on Biologically Inspired Approaches to Advanced Information Technology (Bio-ADIT2004) || Lausanne, Switzerland || January 29-30, 2004
|-
| Seventh International Symposium on Distributed Autonomous Robotic Systems (DARS04) || Toulouse, France || June 23-25, 2004
|-
| Fourth International Worhshop on Ant Colony Optimization and Swarm Intelligence || Brussels, Belgium || September 5-8, 2004
|}

= Summer schools =

{| border=1 cellspacing=0 cellpadding=2
! Name !! Location !! Dates
|-
| Summer School on Co-operative Robotics || Lisbon, Portugal || September 2-7, 2002
|}

= Visits =

{| border=1 cellspacing=0 cellpadding=2
! Name of the lab visited !! Location !! Dates
|-
|[http://www7.informatik.uni-erlangen.de/ Department of Computer Science 7 - Computer Networks and Communication Systems] - Elangen-NÃ¼rnberg University || Erlangen, Germany || November 1, 2005 - Now
|}

= Paper published =

The complete list is [http://iridia.ulb.ac.be/~hlabella/publications.php here].

= Papers submitted =

{| border=1 cellspacing=0 cellpadding=2
! Title !! Journal/Conference !! Download link !! Bibtex link
|-
| Division of Labour in a Group of Robots Inspired by Antsâ Foraging Behaviour || ACM Transactions on Autonomous and Adaptive Systems (TAAS) || ''NA'' || ''NA''
|-
| SWARM-BOT: Design and Implementation of Colonies of Self-assembling Robots || Artificial Intelligence Journal || ''NA'' || ''NA''
|}

= Technical reports =

See [http://iridia.ulb.ac.be/~hlabella/publications.php here].

= Contributions to the lab life =

Not many ;-)

Just a couple of things on the server, the cluster and on a couple of computers....

= Holidays taken during the PhD studies =

Not so many, just a couple of weekends in Germany :-)

History Halva

2005-12-05T11:22:15Z

2005-07-20T09:45:28Z

Hlabella:

Unemployment

2005-07-20T09:36:12Z

Hlabella:

2005-02-17T16:26:58Z

Hlabella: divided the todo list in 3 parts: error/improvements/wishlist

This page contains a list of items which still need to be done on the cluster.

Errors & problems:
* There is a random error when installing the clients in the rack with FAI. The clients start to output a lot of things on screen, but unfortunately they scroll to fast to be read. I could not find any way to block them.
* Neither <tt>yppasswd</tt> nor <tt>passwd</tt> work on the clients of the NIS domain. User have to change password from <tt>majorana</tt>.

Improvements:
* A daemon that checks the status of the UPS should be installed on both majorana and polyphemus.
* Make the configuration of the package on the diskless and on the rack more similar. At the moment FAI takes care only of modifing the important configuation files in /etc.
* Use one repository for the configuation of those packaged which use debconf. I found out that this program can be used to access configuration DBs also shared via NFS of querying a LDAP server.
* Create a script to automaticcally install/upgrade packages on the clients

Wishlist:
* Install Ganglia to monitor the usage of the cluster via web.
* Install LDAP instead of NIS (only if it is better).
* Install a new version of Sun Grid Engine (or something else).
* Install Bugzilla to trace problems on the cluster (and to have a knowledge base of how to solve them!).