# Release Notes for Apache Derby 10.12.1.1

These notes describe the difference between Apache Derby release 10.12.1.1 and the preceding release 10.11.1.1.

- Overview
- New Features
- Bug Fixes
- Issues
- Build Environment
- Verifying Releases

## Overview

The most up to date information about Derby releases can be found on the Derby download page.

Apache Derby is a pure Java relational database engine using standard SQL and JDBC as its APIs. More information about Derby can be found on the Apache web site. Derby functionality includes:

- Embedded engine with JDBC drivers
- Network Server
- Network client JDBC drivers
- Command line tools: ij (SQL scripting), dblook (schema dump) and sysinfo (system info)

Support for Java SE 6 and Java SE 7 is being sunsetted. The 10.13 release family will not support those platforms. The 10.12 release family supports the following Java and JDBC versions:

- Java SE 6 and higher with JDBC 4.0, 4.1, and 4.2.
- Java SE 8 compact profile 2.

## New Features

This is a feature release. The following new features were added:

- ALTER TABLE and identity columns - The ALTER TABLE command can be used to add identity columns now. See the section on this statement in the Derby Reference Manual.
- Cache-monitoring MBean - An MBean has been added for monitoring internal Derby caches. See the description of CacheManagerMBean in the "Introduction to the Derby MBeans" section of the Derby Server and Administration Guide.
- Optional Tool for Handling JSON Data - An optional tool has been added for packing query results into JSON documents and for unpacking JSON documents into tabular result sets. See the section on the simpleJson optional tool in the Derby Tools and Utilities Guide.
- Statistics aggregates - SQL Standard VAR_POP(), VAR_SAMP(), STDDEV_POP(), and STDDEV_SAMP() aggregates have been added. See the "Aggregates (set functions)" section in the Derby Reference Manual.

## Bug Fixes

The following issues are addressed by Derby release 10.12.1.1. These issues are not addressed in the preceding 10.11.1.1 release.

Issue Id | Description
--- | ---
DERBY-6829 | Document the simpleJson optional tool and the SimpleJsonVTI.
DERBY-6825 | Add basic JSON support to Derby.
DERBY-6824 | Move ShutdownException into shared code area
DERBY-6820 | Improve error handling in XmlVTI
DERBY-6807 | XXE attack possible by using XmlVTI and the XML datatype
DERBY-6801 | Implement MessageUtils class so client and server can share message argument encoding/decoding
DERBY-6800 | Implement DerbySQLIntegrityConstraintViolationException class
DERBY-6783 | WHEN clause in CREATE TRIGGER for UPDATE is not working for the sql script below
DERBY-6774 | background post commit threads cause ASSERTS/errors on interaction with alter table add column
DERBY-6769 | sane.derbyTesting.jar.lastcontents can be "out of date" but no build error results
DERBY-6768 | List the enabled protocols in derby.log for network server configuration
DERBY-6753 | Docs for IDENTITY_VAL_LOCAL needs to be updated to indicate that the return value will be impacted by single row UPDATE of identity column
DERBY-6751 | Prevent user code from getting the LanguageConnectionContext from an EmbedConnection
DERBY-6748 | Localize messages introduced or changed in 10.11.1
DERBY-6744 | Update the documentation of security policy files to include the new usederbyinternals SystemPermission
DERBY-6742 | For update statement, collect generated keys if Statement.RETURN_GENERATED_KEYS flag is supplied to the JDBC call.
DERBY-6741 | User code can get the ContextManager from an EmbedConnection
DERBY-6737 | CLOB retrieve exceptions after moving cursor around
DERBY-6733 | Implement an MBean for monitoring caches
DERBY-6730 | Cannot create a Lucene index if a key column's name is case-sensitive
DERBY-6724 | NPE if insert statement needs recompilation after having fired a trigger
DERBY-6722 | GenericStatementContext.cleanupOnError() needs protection from later errors during statement cleanup
DERBY-6720 | Add derbyoptionaltools.jar to the maven artifacts we publish
DERBY-6719 | Add derbyoptionaltools.jar to the class paths of the scripts in the bin directory
DERBY-6717 | Policies with multiple SystemPermissions are not handled well
DERBY-6714 | RuntimeInfoTest failed with insufficient data from server
DERBY-6705 | Triggers should not allow MERGE statements that reference temporary tables
DERBY-6703 | MERGE statement fails with NullPointerException if ON clause references non-existent column
DERBY-6662 | DatabaseMetaData.usesLocalFiles() returns true for in-memory databases
DERBY-6654 | Require that generated code live in the org.apache.derby.exe package.
DERBY-6648 | Application code should not be able to call ContextService.getContextOrNull()
DERBY-6636 | The public api of BaseDataFileFactory may allow blackhats to assume elevated privileges.
DERBY-6635 | OptimizerTracer.unloadTool() could be used to write garbage over Derby data files.
DERBY-6632 | Applications may be able to use StorageFactoryService to delete Derby databases and overwrite service.properties.
DERBY-6631 | FileMonitor can be used to elevate an application's privileges
DERBY-6630 | Applications can use JCECipherFactory to elevate their privileges to those granted to Derby
DERBY-6619 | After silently swallowing SecurityExceptions, Derby can leak class loaders
DERBY-6617 | Silently swallowed SecurityExceptions may disable Derby features, including security features.
DERBY-6592 | Update the version of ant which we tell new developers to use.
DERBY-6569 | NULLIF may return incorrect results if first operand calls non-deterministic function
DERBY-6475 | Update documentation for SYSTRIGGERS after DERBY-5866 changes
DERBY-6414 | Incorrect handling when using an UPDATE to SET an identity column to DEFAULT
DERBY-5466 | Add support for SQL Standard statistics functions, such as STDDEV_POP, STDDEV_SAMP, VAR_POP, VAR_SAMP
DERBY-5165 | Prepared XA transaction locks are not kept across DB restart
DERBY-4057 | Space is not reclaimed if transaction is rolled back
DERBY-3888 | ALTER TABLE ... ADD COLUMN cannot add identity columns
DERBY-3195 | Describe if default security manager & policy is installed or not on each of the mechanisms to start the network server.
DERBY-3005 | Document possibility to specify method signature in EXTERNAL NAME when creating a procedure/function
DERBY-2238 | Example of ScalarSubquery in Derby Reference Manual is not ScalarSubquery
DERBY-2051 | CachedItem's comments and code are inconsistent wrt. syncronization
DERBY-691 | committed deleted row space reclamation may be missed if delete is actually an aborted insert.
DERBY-600 | Document that DB is booted in read-only mode if not able to create db.lck file

## Issues

Compared with the previous release (10.11.1.1), Derby release 10.12.1.1 introduces the following new features and incompatibilities. These merit your special attention.

- Note for DERBY-6807: XML parsing is now performed more securely.
- Note for DERBY-6648: Security policy files must grant a new permission to derby.jar, derbynet.jar, and derbyoptionaltools.jar.
- Note for DERBY-6414: UPDATE statements now accept DEFAULT as a valid value for identity columns.

### Note for DERBY-6807

#### Summary of Change

XML parsing is now performed more securely.

#### Symptoms Seen by Applications Affected by Change

If no Java Security Manager was in place, Derby applications were vulnerable to XML External Entity Expansion attacks (XXE attacks). Such attacks could result in disclosure of sensitive information that the application's user should not have been allowed to view.

If a Derby application used the XmlVTI to parse XML documents, that application was also vulnerable if not protected by a Security Manager policy.

#### Incompatibilities with Previous Release

Applications which depended on the ability to have Derby's XML parser expand external entities may now be unable to use that functionality unless they correctly deploy a Java Security Manager policy authorizing the filesystem access performed by the entity expansion.

#### Rationale for Change

This change was made to prevent any unauthorized information disclosure by the XML parser.

#### Application Changes Required

For detailed information on configuring Derby with a Java Security Manager policy, please see the Derby Security Guide.

### Note for DERBY-6648

#### Summary of Change

Security policy files must grant a new permission to derby.jar, derbynet.jar, and derbyoptionaltools.jar.

#### Symptoms Seen by Applications Affected by Change

Unless this new permission is granted, databases won't boot, the network server won't come up, and the Lucene plugin won't be usable. If Derby runs under a SecurityManager whose policy file doesn't include this new permission, then users will see the following error when booting databases and servers and when using the Lucene plugin:

    java.security.AccessControlException: access denied org.apache.derby.security.SystemPermission( "engine", "usederbyinternals" )

#### Incompatibilities with Previous Release

When Derby is run under a Security Manager, databases and servers won't boot and the Lucene plugin won't be usable unless a new permission is added to the security policy.

#### Rationale for Change

Additional security has been added to Derby. When running under a Security Manager, embedding applications and database routines can no longer access certain sensitive internal structures.

#### Application Changes Required

Users who run Derby under a SecurityManager must edit the policy file and grant the following additional permission to derby.jar, derbynet.jar, and derbyoptionaltools.jar:

      permission org.apache.derby.security.SystemPermission "engine", "usederbyinternals";

### Note for DERBY-6414

#### Summary of Change

UPDATE statements now accept DEFAULT as a valid value for identity columns.

#### Symptoms Seen by Applications Affected by Change

In previous releases of Derby, the following UPDATE statements would raise exceptions:

    create table t1( a int generated always as identity, b int );
    insert into t1( a, b ) values ( default, 100 );
    update t1 set a = default;
    ERROR 42Z23: Attempt to modify an identity column 'A'.

    create table t2( a int generated by default as identity, b int );
    insert into t2( a, b ) values ( default, 100 );
    update t2 set a = default;
    ERROR 23502: Column 'A' cannot accept a NULL value.

The fix for DERBY-6414 makes the above two UPDATE statements work. Now those statements update the identity columns with their next generated values.

#### Rationale for Change

The new behavior conforms to the SQL Standard.

#### Application Changes Required

Applications no longer need to look for exceptions 42Z23 and 23502 when updating identity columns.

## Build Environment

Derby release 10.12.1.1 was built using the following environment:

- Branch - Source code came from the 10.12 branch.
- Machine - Mac OSX 10.7.5.
- Ant - Apache Ant(TM) version 1.9.2 compiled on July 8 2013.
- Compiler - All classes were compiled by the javac from the 1.8.0-b132 JDK, Java HotSpot(TM) 64-Bit Server VM (build 25.0-b70, mixed mode).
- JSR 169 - Support for JSR 169 has been deprecated.

## Verifying Releases

It is essential that you verify the integrity of the downloaded files using the PGP and MD5 signatures. MD5 verification ensures the file was not corrupted during the download process. PGP verification ensures that the file came from a certain person.

The PGP signatures can be verified using PGP or GPG. First download the Apache Derby KEYS as well as the `asc` signature file for the particular distribution. It is important that you get these files from the ultimate trusted source - the main ASF distribution site, rather than from a mirror. Then verify the signatures using ...

    % pgpk -a KEYS
    % pgpv db-derby-X.Y.tar.gz.asc

    or

    % pgp -ka KEYS
    % pgp db-derby-X.Y.tar.gz.asc

    or

    % gpg --import KEYS
    % gpg --verify db-derby-X.Y.tar.gz.asc

To verify the MD5 signature on the files, you need to use a program called `md5` or `md5sum`, which is included in many unix distributions. It is also available as part of GNU Textutils. Windows users can get binary md5 programs from here, here, or here.

We strongly recommend that you verify your downloads with both PGP and MD5.