HYDRA build system
HYDRA build system adminuserBase components of the Make system
The Hades build system is compiled of the following Makefiles:
hades.def.mk
Contains all definitions to build any object
hades.rules.mk
Contains all rules to build a set of Hydra libraries (so called global build)
hades.module.mk
Contains all rules to build a single Hydra library
hades.app.mk
Contains all rules to build an application based on Hydra
Use it to build any library or application in the Hades software environment based on C, Fortran 77, C++ or Oracle PL/SQL. This has nothing to do with loading libraries in Root's CINT - which is a completely different subject. These makefiles are part of the Hydra distribution and saved in the admin directory of the code tree.
HowTo use existing Makefiles
Variables and settings
Before using the Makefiles to build any Hades software, setup the shell environment. The default settings related to third party software are all defined at the beginning of hades.def.mk. These frequently used variables have an influence on building Hydra and related software; they can be set in the Makefiles and on command line, or even in both locations:
Call make with the following targets (without any target, 'all' is taken as the default one):
HADDIR
Absolute path to the directory where Hydra is installed - needed if Hydra is not build completely from scratch. An installation consists of the directories
$HADDIR/lib
and
$HADDIR/include
containing all libraries and their header files as well as directory
$HADDIR/macros
which keeps the related rootlog*.C macros for loading all Hydra modules. In
$HADDIR
itself, the makefiles hades.*.mk are installed.
MYHADDIR
Absolute path to the directory where private and additional Hydra module versions are installed. Also here, an installation consists of the directories
$MYHADDIR/lib
and
$MYHADDIR/include
containing all private libraries and their header files.
BUILD_DIR
Absolute path to the directory where all compiler output files will be located. Default:
./build in case one builds a set of Hydra modules or an application;
../build
in case one build a single module.
INSTALL_DIR
Absolute path to the directory where all libraries and header files will be installed. Default:
Always
.
in case of applications.
For one of several modules:
$MYHADDIR
- if set.
Otherwise
./install
in case one builds modules using a global makefile and
../install
in case one build a single module in the module directory (this refers to the same directory).
USES_RFIO, USES_ORACLE, USES_GFORTRAN, USES_CERNLIB and USES_X11
To build Hydra independent from those 3rd party software set the related flag explicitly to
no
and remove correlated modules in your global Makefile. Otherwise it depends on the default settings defined in the different module Makefiles, which packages are finally used.
Use Cases and Boundary Conditions
Building Hydra completely from scratch:
- Somewhere, there is a directory - let's call it hydra here - which must contain a global Hydra Makefile (which builds all modules), and the hades.*.mk Makefiles. All module directories e.g. base or pid are sub-directories to hydra.
Set HADDIR to hydra, e.g.
export HADDIR=/path/to/hydra
- Unset MYHADDIR - to inhibit mixture of versions, because there shouldn't be private modules, yet.
- These steps are done automatically in the global default Makefile.
Now do:
make; make install INSTALL_DIR=/wherever/you/want/to/have/it
Building a set of private Hydra modules:
- Set HADDIR to the global Hydra installation location, where basically all modules should be installed.
- Set MYHADDIR to your private installation location if you have already some local modules you want to use.
- Write your own reduced global Makefile (see below).
- In the directory, to which the modules to be compiled are sub-directories, do:
make; make install
Building a single module:
- Set HADDIR to the global Hydra installation location, where basically all modules should be installed.
- Set MYHADDIR to your private installation location if you have already some local modules you want to use.
Either do
make; make install
in the module directory, or execute
make MODULES=dir-name; make install MODULES=dir-name
using your global Makefile - MODULES=dir-name is only needed, if you want to override the list of modules which is defined in your global Makefile.
Building an application:
- Set HADDIR to the global Hydra installation location, where basically all modules should be installed.
- Set MYHADDIR to your private installation location if you have already some local modules.
In the application directory, do:
make; make install INSTALL_DIR=/wherever/you/want/to/have/it
, typically
INSTALL_DIR=~/bin
Take care, that you don't mix different library (Hydra) versions while building modules!
- All modules must be sub-directories of the directory which contains the global Makefile (if one was used at all). This Makefile can then be used to build all those modules.
All modules and applications keep dependencies to libraries internally; including the full path as it was found while linking the library ("rpath" mechanism). This can be checked using the
ldd
command.
To use just global modules (from /misc/hadessoftware/install) do:
unset MYHADDIR
- To use just local (private) modules install all in MYHADDIR.
- Local or private modules are always preferred of global ones (MYHADDIR always beats HADDIR).
To temporarily use a module from global location, deinstall it locally:
make deinstall MODULES="dir-name ..."
The re-installation is very fast, since it is just copied from the build directory.
- The hades.*.mk Makefiles are placed in HADDIR by default (see below).
Targets
Call make with the following targets (without any target, 'all' is taken as the default one):
all
Is an alias for 'build'
check
This target is independent of all other targets and checks the filename and directory structure of your project.
depend
Create the dependency files. If this target was not executed, then the files are created during the 'build' step, automatically. Creating the dependencies is much faster than building objects, and will therefore uncover some possible problems much faster.
build
Build all libraries/modules/applications
install
Install the libraries/applications and corresponding header files such, that they are ready for usage by CINT.
doc
Creates the HTML class documentation
deinstall
Antagonist of
install
clean
Deletes all files but dependencies and sources
distclean
Deletes all files but sources
How to write new Makefiles
make -s
Silent Mode - don't echo commands; just explicit echo commands, as well as warnings and errors are shown
make -n
No Operation Mode - echo all commands but don't execute them. Good debugging feature!
make -jX
Parallel Build - process X modules/files in parallel (it turned out, that in case one builds a Hydra library, 2 jobs per CPU/Core is the optimum, otherwise 1 job per CPU/Core is enought)
make -f file
Use 'file' as input makefile
make -p
Print database of variables and rules - nice feature for debugging Makefiles (Hint: Better redirect the bulk output to a file)
- += Append a value to a variable
- ?= Sets a variable to a value only if it's not already set: This gives you the option to override settings via shell environment - variables set on the command line always have the highest precedence.
- := Sets a variable by evaluating the right side immediately
- = Sets a variable by evaluation the right side in the moment the variable is used (this give you the possibility to bequeath changes of settings via different variables)
Example of a global Makefile
MODULES ?= base mdc ora include $(HADDIR)/hades.def.mk ### possibly override default or append new definitons here include $(HADDIR)/hades.rules.mk ### possibly override default or append new rules here |
These lines describe a Makefile which builds the modules 'base', 'mdc' and 'ora'. Each module must have its own Makefile. If one uses '?=' as assignment operator, then one has the possibility to override the list of modules to be build on command line, e.g. with "make MODULES=mdc" just the module 'mdc' is build.
While building Hydra not completely from scratch, you will need at least HADDIR pointing to the correct installation!
The global makefile is a good place to introduce hard-wired private settings. It's better not to change module makefiles, e.g. since temporary changes in several modules might lead to errors if they were forgotten to be changed back or synchronized. Here is an example of some common manipulations of default settings/behaviour:
MODULES ?= base mdc ora BUILD_DIR := /tmp INSTALL_DIR ?= /home/my/install USES_RFIO := yes USES_GFORTRAN := yes USES_CERNLIB := yes USES_ORACLE := yes include $(HADDIR)/hades.def.mk .PHONY: default default: build install include $(HADDIR)/hades.rules.mk |
If you type
make
, this makefile actually does the same things as described above, but it uses /tmp as build directory for all modules, an alternativ installation directory which can be altered via environment settings and does execute the installation automatically after a successful compilation of the modules.
Example of a module Makefile
LIB_NAME := MyLib SOURCE_FILES := file1.cc file2.pc file3.c file4.f USES_RFIO := yes USES_GFORTRAN := yes USES_CERNLIB := yes USES_ORACLE := yes include $(HADDIR)/hades.def.mk ### possibly override default or append new definitons here include $(HADDIR)/hades.module.mk ### possibly override default or append new rules here |
These lines describe a Makefile which builds the library 'libMyLib.so'. This library is made of 4 objects written in C++, Oracle PL/SQL, C, and Fortran 77 (in this order). It uses RFIO explicitly (currently only done by modules base, rfio and htools - otherwise this flag is not needed), parts of CERN library and Oracle. Here, it is not necessary to set USES_ORACLE, since the build system recognizes the usage of an Oracle precompiler file (ending '.pc'). However, it is a good style to use it explicitly if needed.
Example of an application Makefile
APP_NAME := myapp SOURCE_FILES := file1.cc file2.pc file3.c file4.f USES_RFIO := yes USES_GFORTRAN := yes USES_CERNLIB := yes USES_ORACLE := yes include $(HADDIR)/hades.def.mk # override default list of linked Hydra libraries - before they can act on the rules HYDRA_LIBS := -lHydra -lMdc include $(HADDIR)/hades.app.mk ### possibly override default or append new rules here |
These lines describe a Makefile which builds the application 'myapp'. The program is made of 4 objects written in C++, Oracle PL/SQL, C, and Fortran 77 (in this order). It uses RFIO explicitly (currently only done by modules base, rfio, htools and the DST macros - otherwise this flag is not needed), parts of CERN library and only the Hydra libraries 'Hydra' and 'mdc' - otherwise almost all libraries are used by default (see hydra.def.mk). Here, it is not necessary to set USES_ORACLE, since the build system recognizes the usage of an Oracle precompiler file (ending '.pc'). However, it is a good style to use it explicitly if needed.
Software installation locations
Software installation locations adminuserSoftware distribution
The software for the the desktop machines (debian10), lxpool.gsi.de as well as the interactive batch nodes and virgo3 cluster is hosted and distributed via cvmfs (Cern Virtual Machine File System). cvmfs distributes the files to local caches in in each node. When using cvmfs for the first time it can take a while until the files are copied. Once copied, the software should load faster. For debian8 and debian10 software installations for Hydra, Hgeant and more are provided.
From the shell
/cvmfs/hades.gsi.de/install (debian8)
/cvmfs/hadessoft.gsi.de/install (debian10)
shows the installations available at GSI.
The install path keeps the root versions used and in the corresponding versions path
the depending hydra, hgeant and pluto installations. The corresponding environment
scripts (defall.sh) can be found in the application directory. The software is compiled
native 64bit. 32bit compatibility mode is not supported anylonger on the GridEngine
batchfarm.
example:
debian8:
/cvmfs/hades.gsi.de/install/5.34.34/hydra2-6.5/defall.sh /cvmfs/hades.gsi.de/param // parameter files /cvmfs/hades.gsi.de/source // source code of hydra2+hgeant2 for browsing
debian10:
/cvmf/cvmfs/hadessoft.gsi.de/install/debian10/6.24.02/hydra2-6.5/defall.sh
/cvmfs/hadessoft.gsi.de/param // parameter files /cvmfs/hadessoft.gsi.de/source // source code of hydra2+hgeant2 for browsing
Old software
In 2012 hydra was upgraded to hydra2. hydra2 is not compatible to data prior 2012. The
latest ported version of hydra and hgeant has are installed on the /cvmfs/hades.gsi.de
and can be used to work on the current operating systems build ontop of ROOT 5.34.34.
/cvmfs/hades.gsi.de/install/5.34.34/old (debian8)