Advanced Resource Connector (ARC)
=================================

Release Notes for NorduGrid ARC 15.03 update 7          May 25, 2016
----------------------------------------------

This is a minor release, introducing new features and addressing bugs discovered since release 15.03u6.

NorduGrid ARC 15.03 has received an update to:
  * core, clients, CE, Infosys and gridftp - from version 5.0.5 to 5.1.1
  * documents - from 2.0.6 to 2.0.7
Nagios plugins, CAnL C++ and metapackages are unchanged.


New features highlights
-----------------------

  * An option to arc.conf to gather various performance metrics related to A-REX, as specified in
    http://wiki.nordugrid.org/wiki/Logging_of_CE_performance_numbers.
  ** Can be enabled by setting enable_perflog_reporting="yes" in common block.
  ** Default location for performance logs is /var/log/arc/perfdata. Can be changed with perflogdir
     option in common block.
  ** A helper script to upload performance data is partially implemented. Can be invoked by setting
     helper=". /usr/share/arc/perferator" in [grid-manager] (path to perferator may be different if ARC
     is installed from other sources than rpm or deb packages).
  * Improved handling of delegations and proxy renewal, see detailed notes.
  * Added support for Amazon S3.
  ** The URL protocol is s3://
  ** The environment variables S3_ACCESS_KEY and S3_SECRET_KEY should be set to the S3 keys.
  * New URL option "failureallowed" can be set to "yes" and failure to transfer the file will not
    cause job failure.
  * New configuration option "cachedump" in [acix/cacheserver] for creating periodic cache dumps
  * Various harmonizations to logging.
  * Various improvements to the ARC SDK.
  * Numerous bugfixes, enhancements and improvements.

Detailed notes
--------------

== ARC Server and core components

The following issues were fixed in the ARC core:
  * Added performance metrics for A-REX and data components.
  * Improved handling of VOMS and VOs.
  * Added gm-jobs option to obtaion delegation for a specified job.
  * Added delegation handling to WS interface.
  * New optional technology preview to use sqlite rather than bdb as delegation DB. Can be enabled
    with delegationdb=sqlite in [grid-manager]. Please note that jobs should be drained before
    switching between DBs.

=== Accounting

  * No changes.

=== Information system

The following issues were fixed in the information system:
  * Added performance metrics for infosys based on the PERL Devel::NYTProf profiler available from
    EPEL, Debian/Ubuntu repositories and CPAN.
  * Changed log format to be consistent with A-REX logging.

== ARC Clients

The following issues were fixed in the ARC Clients:
  * Enhanced job description parsing.
  * Made it possible to change UserConfig object for SubmitterPlugin and JobControllerPlugin classes.
  * Created structure for mapping policy and retrieve information from GLUE2.
  * Enhanced job description parsing and assembling error reporting.

== Nagios plugins

  * No changes.

== Common authentication library CaNL++

  * No changes.



Fixed bugs
----------

Since ARC 15.03 update 6, the following bugs were fixed:

  * 3414 Request for job logging
  * 3477 sessiondir = path [drain] can't prevent new jobs entering sessiondir
  * 3479 Description of voms-based authorisation is wrong
  * 3486 External helper log file location is hardcoded to controldir/job.helper.errors
  * 3492 certificate generated with arcproxy is not usable with arcls/arccp + dCache SRM SE - one with
         voms-proxy-init is
  * 3496 arcplugin occasionally segfaults
  * 3520 emies maps to wrong uid
  * 3522 Renewing proxy through EMI-ES doesn't work with dynamic output files
  * 3524 Encoded characters in location break HTTP redirection
  * 3525 Wrong ARC version number in /usr/include/arc/ArcVersion.h
  * 3529 arc-lcas and arc-lcmaps helpers segfault when validating voms credentials of VOs whose VOMS
         certificates are signed by "/DC=com/DC=DigiCert-Grid/O=DigiCert Grid/CN=DigiCert Grid CA-1"
  * 3535 URL option for opportunistic output file upload
  * 3537 Unable to submit jobs with EMI ES interface only
  * 3539 Submission to PBS backend with memory limit does not work well with threads
  * 3542 %R substitution in auth plugin uses first session directory
  * 3543 Invalid input file URL can make A-REX crash
  * 3552 Restarting A-REX causes FINISHING jobs with dynamic outputs to fail


Known issues
------------

  * There is a memory leak when using Java API for multiple job submission with files to BES interface.
  * The CPU time is not measured correctly for jobs that kill the parent
    process, such as some agent-based/pilot (e.g., ALICE)
  * JURA will not publish records to the APEL on a standard Debian/Ubuntu
    system, because the python-dirq package is not available for them. The
    workaround is to build this package from source
  * When using ARC client tools to submit jobs to CREAM, only JSDL can be used
    to describe jobs, and the broker type must be set to Null in the client.conf
    file
  * ARC GUI (arcjobtool) is not available yet, pending implementation of client
    library changes
  * Standalone client tar-balls for Linux are not yet available
  * Bug 2905 is solved using workaround. Source of problem is not yet
    identified
  * A-REX can under some circumstances lose connection with CEinfo.pl and go into an infinite loop. The
    only current workaround is to restart the a-rex service.
  * twistd, the underlying engine for ACIX, sometimes logs into rotated ACIX log files.
    While all log messages are correctly logged in the main log file, some rotated log
    files may receive new log messages.
  * submit-*-job do not have permission to write performance metrics to log.


Availability
------------

= Source

ARC release 15.03u7 consists of the following source packages:
  * NorduGrid ARC, version 5.1.1 (main components)
  * NorduGrid ARC Documents version 2.0.7
  * metapackages for client tools, computing element and information index,
    version 1.0.7
  * Nagios probes for ARC CE, version 1.8.4
  * gangliarc - ARC Computing Element monitoring in ganglia, version 1.0.0
  * Common authentication library caNl++, version 1.0.1

Source code for main components is available from here:
  http://svn.nordugrid.org/repos/nordugrid/arc1/tags/5.1.1

Documentation source (mostly LaTeX) is available from:
  http://svn.nordugrid.org/repos/nordugrid/doc/tags/2.0.7

Source for metapackages is available from:
  http://svn.nordugrid.org/repos/packaging/{fedora,debian}/nordugrid-arc-meta/tags/1.0.7

Source for Nagios probes is available from:
  http://svn.nordugrid.org/repos/nordugrid/nagios/tags/release-1.8.4

Source for the common authentication library caNl++ is available from:
  http://svn.nordugrid.org/repos/workarea/caNl++/tags/1.0.1

Source for gangliarc is available from:
  http://svn.nordugrid.org/repos/nordugrid/contrib/gangliarc/tags/1.0.0

= Repositories

See detailed description at:
  http://download.nordugrid.org/repos.html

These repositories provide binary packages for:

  * Debian: 5.0, 6.0, 7.0 and 8.0 (i386 and amd64)
  * Fedora: from 3 to 23 (i386 and x86_64)
  * CentOS: EL4, EL5, EL6 and EL7 (i386 and x86_64)
  * Ubuntu: 8.04, 8.10, 9.04, 9.10, 10.04, 10.10, 11.04, 11.10, 12.04, 12.10, 13.04,
    13.10, 14.04, 14.10, 15.04, 15.10 and 16.04 (i386 and amd64)

Scientific Linux and RedHat are implicitly supported through corresponding
CentOS repositories.

Microsoft Windows and Mac OS X versions are available from same repositories for
clients and some services.
