----------------------- OrangeFS Release ChangeLog ----------------------- orangefs-2.10.0 (Git Branch v.2.10.0) https://github.com/waltligon/orangefs.git up to commit-id 7c28362b New functionality: * - general updates 1. numerous autotools related m4 files are updated. 2. code refactored in numerous places to eliminate compiler warnings from new versions of gcc. 3. documentation updates. * - Server 1. updated formatting to conform to standard in simple-stripe code. * - Clients 1. build liborangefs by default. liborangefs provides an orangefs version of posix system calls. 2. Windows Client * Retargeted Client to the user mode file system library "dokany" v1.5.1.1000. (See https://github.com/dokan-dev/dokany) * Retargeted build to Visual Studio 2022. * Updated Event Logging for modern versions of Windows. * Removed obsolete security modes. * Updated the executable installer to install dokany and necessary Visual C++ runtime libraries. * Updated OpenSSL. * Made several quality-of-life improvements. * - bmi 1. Eliminated IBV_EVENT_WQ_FATAL warnings 2. Added a new RDMA module: * Utilizes the RDMA Communication Manager library (librdmacm), abstracting transport-specific details out of connection management * Adds RoCE v2 support to BMI 3. Added a new experimental InfiniBand module for future development: * Still a work in progress and minimally tested * Current experimental changes include: - Improved connection-based reference counting * - general fixes 1. Fixed warning about using fast networks and TCP in pvfs2-genconfig * - Client fixes 1. Windows Client: * Fixed a failure to delete files in some cases. * Fixed compatibility issues between dokany 1.5.1 and the prior version. * Fixed issues with error codes and directory-handling. ----------------------- Development 2.9.9 orangefs-2.9.8 (Git Branch v.2.9.x) https://github.com/waltligon/orangefs.git up to rev 65ab0d2 This version of OrangeFS includes many bug fixes. 1. Disable usrint by default. 2. Fix potential memory overwrites in various utilities. - Client-core - Showcoll - Mkspace - Xattr - Viewdist - Str-utils 3. Eliminate compiler warnings. - State machines - Trove 4. Improve parsing of config file. 5. Update documentation flow. 6. Updated readmes. 7. Migrate documentation to hugo format. 8. Update perf mon example. New functionality: 1. Added RoCE v1 support to our IB suite. * - Server 1. Fixed problems with crdirent and rename when using distributed directories. 2. Fixed intermittent problems with updating timestamps. * - Clients 1. pvfs2-stat.c When the object is a symlink, we now display the string length of the target in the size field, making the output consistent with the linux kernel. Corrected call to sys-lookup so that symbolic links are followed correctly when -L is specified on the command line. 2. System Interface 1. Fixed problems with crdirent and rename when using distributed directories. 2. Fixed intermittent problems with updating timestamps. * - BMI 1. Part of adding RoCE v1 support was to make the IB port used for communication configurable. * - Software Infrastructure * - Testing 1. Generic tests, #467 and #477, of the xfstest suite were removed from our testing because we do not support the functionality. 2. Updated our testing to use version 20170929 of LTP. Updated the list of testcases to be used. * - Kernel Module orangefs-2.9.7 (Subversion Branch v.2.9.x) http://dev.orangefs.org/svn/orangefs/branches/v.2.9.x up to rev 13115 in trunk New functionality: 1. Added support for Oracle Linux 7.3 with one caveat: Cannot use OrangeFS as the backing filesystem for a user-defined loop device. Support will be forth coming. 2. Added support for large LMDB databases using DBMaxSize config parameter. 3. Added scripts in the examples/profile.d directory that can be placed in /etc/profile.d. These scripts add OrangeFS locations to LIBRARY_PATH, C_INCLUDE_PATH, AND LD_LIBRARY_PATH. 4. Added support for Fedora 26. Certificate security is not yet compatible with OpenSSL 1.1 but work is in progress. 5. Official OrangeFS rpm is in the Fedora distribution. The rpm will be in either Fedora 27 or Fedora 28. The rpm will also be in their EPEL repository, allowing other RHEL distributions to access it. 6. Added --with-lmdb to the configure script, so you can specify the location of your LMDB installation. Otherwise, LMDB delivered with OrangeFS will be used. NOTE: Either way, to use LMDB instead of Berkely DB, you must specify --with-db-backend=lmdb on the configure line. 7. pvfs2-genconfig will ask you where you want to put the server config file, with the default being the /etc/orangefs.conf. You can still put the server config filename directly on the command line. 8. ACL processing is now done as specified by POSIX. * - Server * - Enhanced permission checking and job scheduling for tree operations. * - Removed memory leaks from the getattr operation and restructured keyval buffer management. * - Corrected problem with mgmt-create-root-dir related to having a correct capability when calling tree-setattr. * - Fixed capability errors that may occur during mkdir. * - Correctly populate the position token during a readdir operation to prevent overrun of dirdata index bits. * - Correct the max size calculation for a readdir response. * - On some platforms, issuing a pvfs2-ls would cause one or more pvfs2-server processes to segfault due to an invalid parameter sent to the server when list-attr was called. This problem was discovered when the code is compiled using the Fedora-specific rpm requirements. * - Made atime on directories more POSIX complient, i.e., we do not update atime on a directory when an entry is created or removed. * - Don't update ctime on file access (xfstests generic/003). * - Clients * - Client Core * - Enhanced the readahead cache with multiple readahead buffers. * - Corrected double-free bug in readahead code * - Corrected additional problems with readahead code and removes memory leaks identified by Valgrind. * - Enhanced the remount procedure to handle new PVFS2_VFS_OP_FEATURES request. * - Corrected problem where client-core went into an infinite loop when a file create operation detected that the file already exists and hints were recreated for a call to lookup. It was the hints processing that was in the infinite loop. * - All Clients * - Dual license OrangeFS JNI and Hadoop clients under both the LGPL version 2.1 and Apache License version 2. Added Omnibond to the Copyright in source files where appropriate. * - Corrected readdir token processing that periodically caused directory entries to be ignored. * - Enhanced error message in msgpairarray_completion_fn. * - BMI * - InfiniBand protocol * - Improved IB's ability to report a lost connection, allowing the system interface to recover the connection. Needed for failover. * - Added warning to pvfs2-genconfig when multiple BMI modules are activated in the server config file. This warning indicates that having more than one BMI module enabled at runtime will degrade performance. NOTE: You can safely compile multiple BMI modules at configure time, but you don't want to activate multiple modules in your server config file, unless your situation demands it. * - Software Infrastructure * - Improved Makefile processing for parallel make. * - Get prepare to fail more gracefully if automake is not installed. orangefs-2.9.6 (Subversion Branch v.2.9.6) http://dev.orangefs.org/svn/orangefs/branches/v.2.9.6 http://dev.orangefs.org/svn/orangefs/tags/v.2.9.6 New functionality: 1. Added an user application, ofs_setdirhint, enabling one to set directory hints. See the OrangeFS Wiki page for more details at http://dev.orangefs.org/trac/orangefs/wiki/Directory Hints. 2. Added readahead caching in the client core. See the OrangeFS wiki page for more details at http://dev.orangefs.org/trac/orangefs/wiki/Client Core Readahead Cache. 3. Added support for Linux kernel 4.1.6 on Fedora. 4. Added kernel module support for iter vs iov/nr-segs for kernels that support iter. 5. An additional package is required to build OrangeFS: libattr-devel. * - Software Infrastructure * - Added malloc tracing feature. * - Corrected issues with the --enable-fast option. This option compiles out gossip-debug statements and asserts, and sets the gcc optimization level to 3. * - Testing * - Excluded test 360 from xfstests. * - Modified mpi-active-delete.c to handle FILE-NOT-FOUND on an unlink, increasing sleep time for the rank 0 process whenever a FILE-NOT-FOUND situation presents itself, and increased the number of delete tries from 5 to 10. * - Client * - ALL CLIENTS: * - Modified pvfs2-cp so we don't try to change the number of dfiles on an existing file. * - Improved acache processing by getting all attributes when only asked for a few, including capability. * - Invalidated acache attributes for a directory when objects are created or removed. * - Changed logic in the msgpairarray completion function in anticipation of a future change where a capability is invalidated from the capcache when the server says the capability has timed out. * - Change the implementation of the token returned by readdir, negating the need for an rcache. * - Removed memory leaks. * - Added an user application, ofs_setdirhint, that sets directory hints. * - CLIENT CORE: * - Handle mount command error when URL doesn't have a port number. * - Added readahead caching feature. * - Changed readahead parameter into union so as not to change kernel protocol. * - Added support allowing the upstream Linux/OrangeFS kernel module and the client core to communicate a "features" list between them, in order to support backwards compatibility with future enhancements. * - Kernel Module * - Changed two-parameter kernel op from int64_t to int32_t to match one-parameter kernel op. * - To make the kernel compile with Fedora with Linux version 4.1.6-201, added support for iter vs iov/nr-segs. * - Cleanup of iov code in pvfs2_devreq_writev, uses copy_from_iter instead of copy_from_user on kernels that support iter. * - Corrected khandle-related memory leak. * - Fixed lock inversions in remount code. orangefs-2.9.5 (Subversion Branch v.2.9.5) http://dev.orangefs.org/svn/orangefs/branches/v.2.9.5 http://dev.orangefs.org/svn/orangefs/tags/v.2.9.5 This version of OrangeFS includes many bug fixes and some new functionality. 1. Full support for Infiniband with our 2.9 system. 2. The 2.9 kernel module is now part of Linux upstream, beginning with kernel 4.6. Available at kernel.org. 3. Support for LMDB as a metadata store. 4. Replaced karma with a Grafana solution. 5. Kernel module delivered with OrangeFS allows O_DIRECT in open. 6. Kernel module delivered with OrangeFS is compatible up to Linux Kernel 4.4. * - Software Infrastructure * - Enhancements to redefined malloc functionality. * - Added helper function to free attr objects. * - Comments and small format changes designed to improve understandability of locks in the client code. * - USRINT libraries (libofs,liborangefs,liborangefsposix) * - Enhancements to ofs_rm and ofs_cp. * - Enhancements to stdio.c. * - Minor bug fixes. * - Security * - Modified how a client determines expiration of a capability, allowing for network transfer time. * - When sending a small amount of data across the network that results in a capability timeout, automatically regenerate a new capability and try again. The retry was causing the wrong data to be stored on the server. Removed memory leaks. * - Client * - ALL CLIENTS: * - Added check in I/O state machine to automatically renew a capability when it has expired. * - Corrected a memory leak found in the remove state machine. * - Prevent permission errors when using truncate to extend the size of a file. * - Handle extended attribute values up to 8K in length. * - HADOOP: * - default Hadoop version is now 2.7.2. * - Server * - Modified the batch-create process so the capability has a ten-year timeout. * - Increased the default pool sizes to 1024 for metadata, datafile, direcotry, and dirdata handles. * - Increase the PrecreateLowThreshold for metadata, datafile, directory, and dirdata pools to 256. * - Kernel * - Corrected a list-add corruption error seen in the kernel logs when the OrangeFS request-list and in-progress tables are accessed. * - Returns the correct number of bytes received after a readdir call. * - Added ifdef for HAVE_KCONFIG. * - Modified pvfs2_follow_link due to changes in Fedora 23. * - Added write-iter support for the newer kernels. * - Strengthen sanity checks in PINT_dev_write_LIST. * - Fix getdents for both filldir and dir_emit. * - Client Core: Fix khandle related problem with EEXIST CREATE case. * - Added support for the 4.4 kernel. * - Removed deprecated sys/kdaemon.h header from bdflush01.c in LTP per http://lists.linux.it/pipermail/ltp/2016-January/000704.html. This is necessary for LTP to compile on Ubuntu 16.04. * - Added support for O_DIRECT in the open call. * - BMI * - Native IB: Now available. * - Native IB: check for existence of experimental DCT event types and compile around them. * - ALL METHODS: Corrected a segmentation fault that can occur when encoding a listattr response for network transfer. * - Trove * - Added support for LMDB 0.9.18. * - Added configure option for LMDB: --with-db-backend=lmdb Default is still Berkeley DB. Must create a new storage area. A migration path from BDB to LMDB is not provided. * - Handle DB_BUFFER_SMALL errors in Berkeley DB. * - Changed error messages that referred specifically to Berkeley DB. that were in modules that are used with any DB. * - Fix memory leak found in Berkeley DB code. orangefs-2.9.3 (Subversion Tag v.2.9.3) http://dev.orangefs.org/svn/orangefs/tags/v.2.9.3 * - USRINT libraries (libofs,liborangefs,liborangefsposix) * - Minor bug fixes to ofs_cp and ofs_rm * - Added long version of options to ofs_rm * - Resolved compilation errors associated with ofs_cp * - Kernel Module * - Modifications associated with upstream kernel module that future proof the code. * - Corrected an ACL issue. * - Corrections to the /proc/sys/pvfs2 perf counters. * - Modifications allowing Linux 4.0 to compile * - Configure * - Modifications allowing for Linux 4.x * - Client/Server (Linux) * - New functionality to admin app pvfs2-cp. Will allow file to be created without write permissions but can be written by the user who created it. * - Windows Client * - Corrected issue where a memory fault occurred during an I/O operation. * - Added a workaround for credential caching timeout, until a more robust method can be designed. orangefs-2.9.2 (Subversion Tag v.2.9.2) http://dev.orangefs.org/svn/orangefs/tags/v.2.9.2 * - Client/Server (Linux) * - Automatically disabled capability caching, server side, when default security is used. * - Added TreeThreshold option in the server config file. * - pvfs2-genconfig generates a config file with distributed directories turned off, by default. * - Added lazy atime support. By default, atime for a file will be updated upon a read or write if that atime has not been updated within the last 24 hours. This behavior can be modified by setting the environment variable, PVFS2_RELATIME_TIMEOUT, before starting the client: PVFS2_RELATIME_TIMEOUT < 0, disable atime updates for reads/writes PVFS2_RELATIME_TIMEOUT == 0, update atime for every read/write PVFS2_RELATIME_TIMEOUT > 0, update atime for a read/write if atime has not been updated within the last PVFS2_RELATIME_TIMEOUT seconds. * - Added support for mixed endian environments for distributed directories. * - WebDav Client * - Enabled PAM support. The README file, in src/client/webpack, describes how to use this feature. * - USRINT libraries (libofs,liborangefs,liborangefsposix) * - Modified selinux related files to allow compiles on both new and old RHEL distributions. * - __getdelim(): caller can now pass in a buffer pointer allocated by pint-malloc OR regular malloc. NOTE: PINT-malloc is an internal macro that reduces the likely hood of memory leaks. * - Added new calls to PINT-malloc that checks the validity of a pointer's address and determines if PINT-malloc was used to allocate its memory. This call is costly, making two system calls. Use only if necessary. * - Added two new programs, ofs_rm.c and ofs_cp.c, that use the usrint libraries and FTS. These programs are still experimental but can be used as a guidline for building your own programs using the usrint libraries. They are stored in src/apps/user and are currently NOT installed in /bin when a make is issued. * - HADOOP * - Modified build-and-install files to gather OrangeFS version information during configure time. * - OrangeFS is now available for both Hadoop 1.0 and 2.0. See documentation for further details. * - BMI and Native IB (in development) * - Removed loopback from distributed directories code in the mkdir.sm state machine. * - KERNEL MODULE * - GOSSIP: changed GOSSIP_PROC_DEBUG to GOSSIP_CLIENTCORE_DEBUG * - DCACHE: corrected problem where negative dentries did not have the d_op field filled in with the OFS dentry operations functions, causing the kernel to return file-not-found when the file was created outside of the kernel. * - Add configuration check for f_dentry in struct file, allowing for compatibility with newer kernel versions. * - Modified the handling of xattrs, i.e., the exact length of the xattr's value will be correct, not length + 1. orangefs-2.9.1 (Subversion Tag v.2.9.1) http://dev.orangefs.org/svn/orangefs/tags/v.2.9.1 * - Update pom.xml files to use the latest versions of our dependencies * - Add support for issueing -rf with the pvfs2-server program to indicate * you want to have your data/metadata directories on that server wiped * clean and reinitialized in one invocation of pvfs2-server * - Fixed problem that sometimes occurred when reaching the threshold for * splitting distributed directory entries to another server * - Fix for attribute changes to directories not being stored * - Fix compile errors with pvfs2-client-core that occurred with certain * --disable parameters specified at configure time * - Fix compile error experienced on ARM architectures * - Add support for new environment variable for use with Hadoop: * ORANGEFS_STRIP_SIZE_AS_BLKSIZE * - Use "struct ctl_table" rather than relying on the typedef as it went * away around kernel version 3.16. * - Get rid of the lazy mtime flag in the kernel module as the iattr already * has the timestamp * - Fix configure test for O_DIRECT Up to Subserversion Rev 11536 * - Implemented key-based and certificate-based security modes * - Implemented distributed directories * - Added support for Hadoop v2.x * - Updated kernel module for newer kernels * - Allowed root to create file with arbitrary owner and group * - Properly handled variadic arguments passed to open/open64 and * openat/openat64 * - Improved JNI debug flexibility * - Added support for -r and -f flags to pvfs2-rm * - Updated sys-create to populate acache for increased performance * - Fixed minor bugs in DAV module * - Made develtools no longer dependent on pvfs libraries Up to Subversion Rev 10730 * - JNI Layer: * - Removed classes that are no longer used. * - Minor bug fixes. * - USRINT libraries: * - Files created with a zero mode can now be truncated. * - /dev/shm areas are automatically deleted. * - Kernel Module: * - Linux dcache entries are automatically revalidated to prevent stale handles. * - If interrupt occurs while issuing a pvfs2-getattr, EAGAIN is returned * - instead of marking the inode bad. * - Admin Apps: * - pvfs2-write: allows users to run I/O tests without reading from local * - disk. * - Admin Scripts: * - pvfs2-start-all and pvfs2-stop-all: * - Starts and stops servers. * - Located in the sbin install directory * - * - Updated kernel module to work with kernel versions up to 3.11 * - Updated pvfs2-touch to respect umask * - Improved multi-fabric BMI support * - Fixed problem with FUSE readdir code. Up to Subversion Rev 10349 * - Fixed bug in iocommon_open where the variable directory * was used instead of the argument path before calling iocommon_expand_path * - Fixed bug in closedir where the fd was never getting closed thus * leaving lots of fd's open in some cases * - Fixed bug in is_pvfs_path where argument path was referenced without * a dereference (not sure how that ever worked) * - BMI_add_to_socket_collection and related macros were not checking * for errors from epoll_ctl, some are expected but errno was not reset * leaving to issues later on * - Made FUSE compile on MAC OSX 10.7.5 * - Corrected a deadlock issue where the client-core was trying to get a * "write" semaphore at the same time the kernel was holding a "read" * semaphore while sleeping. * - Set PVFS2_DEFAULT_SLOT_TIMEOUT_SECS from 1800 to 900, which means that * when all the shared memory slots are busy, the process will sleep for * 15 minutes (instead of 30 minutes) or until a slot becomes available. * - Made pvfs2-genconfig default the log file location to * /var/log/orangefs-server.log. * - In the OrangeFS extension to Hadoop's abstract FileSystem class, * corrected scenario where an exception was being throw as a result of * mkdirs failure. This error was thrown when client A checks to see if * a parent directory exists, but client B successfully creates the parent * directory prior to client A executing mkdirs. If mkdirs returns false, * then client A should check to see if the directory actually had been * created before throwing an exception. * - Allow mmap() calls with MAP_SHARED * - Fixed several Infiniband-related issues: * - Fixed per connection memory leak. * - Replaced asserts with error checking. * - Changed error*() calls to not exit. All calls to error*() now do * error handeling. * - Lowered QP timeout from 5 minutes to 17 seconds. * - Fixed problem in rewinddir * - Improved how we expose directory entries to Hadoop using * java/util/ArrayList * - patched a problem in iocommon_lseek where dirs over 1024 entries can * cause a problem - also avoided a slow loop when we can. * - Corrected bug related to recursive_delete when using Hadoop * - Added flags to javah javac and jar commands to limit the ammount of * heap memory reserved for jvm. Before, running 'make' with the option * '-j9' was causing multiple jvms to each acquire 1/4 of the machine's * ram by default. Since 9 jvms cannot each obtain 1/4 of a machine's * memory, errors were thrown. Setting a reasonable maximum heap reserved * for jvm fixes this. 256MB works well for now. * - Added warnings for meta/data stored on a root device * - fix selinux issues within usrint * - Added shared descriptor tables, that work with fork and exec, and added * some flags to the link line to improve interposition. * - Corrected issue with JNI layer and gcc not working well without -O0 * option. * - Significant improvements to JNI layer and Hadoop extensions. * - added enable option to build libofs-s.so with -Bsymbolic to force * programs to call it and not another version of the libc routines. * - added an instance of each of two structs of pointers to methods for * system calls and library calls respectively so that external code can * force a call to our code. * - fixed bug in rename to return 0 if try to rename something to itself * - reworked IO locking to use the NPTL stuff * - added missing read of two funcs to stdio_ops * - changed variable named shmctl to shmctrl to avoid a name conflict * - fixed a problem in userint.h where a define was mis-named * - added error.c and error.h from glibc so they link with usrlib Rev 10182 * Reworked usrint initialization to support fork/exec effectively, * particularly for shells (like bash). File descriptors were moved to * a Posix shared memory object so that they can be passed from parent * to child in a fork and from one program to the next in an exec. Fork * creates a new shm object and copies the parent's into it. Exec takes * over the previous program's object but must modify internal pointers. * program loading OFS usrint for the first time creates and initializes * a new object. File descriptor status structs can be shared from * parent to child on a fork. Shared synchronization is used to manage * access to these objects. Management of free space in the shared * objects is done in unshared user space memory using fairly simple * free lists. * * Added features to gen_locks to initialize recursive and shared mutexes * and shared conds for use in the new usrint features. * - Incorporated patch to fix a memory corruption issue * - Fixed client initialization issue resulting from an uninitalized * ibv_wc struct. * - Added a thread for the TCP accept processing for the IB connection * information exchange. * - Fixed the RTS_DONE missing errors * - Updates to JNI layer. Now using NIO OutputChannel for buffered output. * - Added distribution options to pvfs2-genconfig. * - Added pvfs2-start-all.sh convenience scripts that allow an administrator * (with password-less ssh configured) to start or stop the pvfs2-server * on nodes listed in the pvfs2 configuration file. * Run with -h or --help to see available options. * - added cross platform client library, windows client GUI app, and * VS 2012 projects * - Corrected bugs in the OrangeFS extension of Hadoop's abstract * FileSystem class. There were issues with mkdir and delete. * - Changed the way reserved range params are interpreted in pvfs2-genconfig * - OrangeFS client JNI require --enable-jni and --with-jdk at configure * time. OrangeFS extensions to Hadoop classes require --enable-jni, * --with-jdk=, --enable-hadoop, and * --with-hadoop=. After the 'make install' step of * OrangeFS installation: run 'make jni' and 'make jni_install' to build * and install the client JNI. If building Hadoop extensions as well, then * just run 'make hadoop' and 'make hadoop_install'. * - Changed default log file names to use "orangefs". * - Added --disable-opt option to ./configure to disable compiler optimization * - Updated Hadoop/JNI Makefile to reflect latest stable version of Hadoop (1.1.2) and OrangeFS installation path described in new documentation (/opt/orangefs). Also corrected a bug where function dup3 was being defined twice. Rev 10075 * Added support for glob in usrint * * Fixed several bugs that involved using an uncleared buffer that becomes * corrupted. * * Added a malloc and associated functions wrapper that uses #defines to * replace malloc with PINT-malloc(). PINT-malloc() uses glibc malloc() (and not * a client program's replacement for malloc) to do memory management. Inserts * a magic number before each allocated buffer that is check on free to prevent * a number of common malloc related bugs. Zeros newly allocarted buffer. * Zeros buffer being freed. Can be adapted for other uses as well. Includes * malloc debugging calls (currently controlled distinctly from gossip but * will be integrated into gossip). Configure options can turn off some or * all of these features. Also incAudes a set of functions clean-malloc() that * calls to the default version of malloc(). This makes pint-mem.c obsolete. * * Began migrating to a new initialization mechanism that uses a feature of * GCC __attrubute__((constructor)) that causes a routine to run before main. * This should enable us to remove the many calls to init routines currently in * the usrint code. May not work on other compilers, but there may be an * older alternative. Worst case we replace the many calls with macros. A * similar destructor attribute exists for shutdown * * Reworked implementation of stdin, stdout, stderr so that we can use them * effectively with freopen and OrangeFS files. Reworked freopen to support * this. Can still be disabled with configure. * * Made some changes to handling of locks in streams. * - Added OrangeFS Direct Client JNI layer and Hadoop related extensions. * See 'src/client/jni/README' for more information. * - subtle bug when storing directory paths was getting garbage that later * come back to hurt if just the right conditions existed. Went over all * code that worked on pd->s->dpath to ensure everything wil be clean. * - changed nocache hint to cache hint and moved from create to open * - fixed problem with stat by clearing buffer before the sys call * - Minor changes to PINT_string_rm_extra_slashes_rts. * - added glob.c to libofs in case anyone needs it * - changed st_blocks to be size/512 as it should be leaving st_blksize alone * - Improved the output of pvfs2-ls when using the recursive option: 'R'. * Extra slashes were appearing. Wrote a function to strip multiple slashes * and optionally remove the final trailing slash. * - fixed problems found by Jim Kusnier * iocommon_getdents clear buffer before reading dirents * iocommon_getdents uses strncpy rather than memcpy * iocommon_getdents always returns unknown d_type * iocommon_open reworked so it is much easier to understand and less * error prone * reworked calls to expand_path so they get called when they should * tweeked PVFS_sys_lookup to return propoer error codes * fixed fcntl dup command to work correctly (not like dup2) * - fixed error code when xattr is not supported * - fixed some CPP code determining size of a pointer * - Modified the include directive for fuse from to . * pkg-config will correctly identify where the include directory is * located for fuse. (pkg-config --cflags fuse). orangefs-2.8.7 (Subversion Tag v2.8.7) Revs 9399 thru 9763 * WARNINGS: * - The pvfs kernel module is NOT compatible with linux kernel versions * 3.4 and higher. Work is currently underway. * - A known linux kernel bug causing kernel panics when a PVFS * filesystem is unmounted was found in RHEL releases 2.6.18-308.1.1 thru * 2.6.18-308.20.1. The OrangeFS team recommends using 2.6.18-308 (RHEL 5.8) or * 2.6.32-71 (RHEL 6,0) and up. This kernel bug affects other * filesystems as well. * Corrected issues with the readdirplus operations effecting the dokan interface for Windows. * Corrected memory overwrite that resulted from using both readdir and readdirplus members of a union whenever the readdir state machine was called from the readdirplus state machine. * Starting in linux kernels 3.4, d_make_root is used instead of d_alloc_root. Configure changes were added to account for this change in the kernel module. * Fixed a bug in pvfs_readdir associated with the usrint library. * Modified kernel module to accommodate the RCU walk initiated by the system in pvfs2_d_validate. This fixes a deadlock occurring in kernel versions 2.6.38 and later. * Fixed bug in usrint library where a dir is opened and the fd is passed to fchdir but the path is missing. * Added support for fts interface in usrint library. * Fixed errors with invalid objects in the kernel module. * Performance improvements for Windows Client. * Defects corrected for ucache system. * Changes to Makefile: * - Using glibtool instead of glib on MAC-Darwin OS when making libpvfs2.a * - Ensure that sbin dir is creatd before moving ucache system into it. * - Added lib/libpvfs2.so as a dependency when making libofs.so to allow * make with the -j option to operate correctly. * - Renamed liborange to liborangefs. * New Feature: added human readable times to pvfs2-db-display. * New Feature: when setting the mode (admin or normal) of a filesystem using pvfs2-set-mode, you can now do so for individual servers. Prior to this change, the mode was set for all the servers within a particular filesystem. orangefs-2.8.6 (Rev v2.8.6) * New Feature: users can now specify at configure time how they want the * kernel module to handle the file pointer when an error occurs. The * default behavior is to leave the file pointer alone, i.e., the file * pointer will always represent the position of the last byte written, * even though the user's entire request is not totally satisfied. In * this case, a the actual number of bytes written is returned to the user. * The new feature will change this default behavior such that the file pointer * is repositioned to the byte just prior to the user's write request, a * -1 is returned to the user's call, and errno is appropriately set. To * enable this feature, add --enable-reset-file-pos on the configure * command line when your OrangeFS system is configured. * Bug fix: modified pvfs2_readdir, which is executed by the kernel module * on behalf of an "ls", to properly retry or quit when pvfs2-client-core * aborts/restarts or when pvfs2-client terminates. Prior to these * changes, pvfs2_readdir could possibly corrupt another process's "ls" * command by releasing access to a shared memory buffer index that may * be legitimately in use. * Bug fix: modified the kernel code to re-post data to the shared memory * system (for I/O requests) whenever pvfs2-client-core aborts. Because the * data was not re-posted prior to these changes, the resulting file contained * corrupted data. * New Feature: added hint support to pvfs_open() to allow user to * provide various things like the number of servers to use, the * distribution to use, the layout to use, as well as tags that can be * used to trace requests. * New Feature: added new layout LOCAL which puts a file on the server on * the same node as the client, if there is one. Otherwise defaults to * ROUND_ROBIN * Bug Fix: repaired a bug in the layout processing * Bug Fix: fixed some problems with void pointers and replaced a missing * function prototype * Bug Fix: Corrected a long time bug: if the number of unexpected requests defined in the config file is greater than 64, then the code inadvertently used invalid job_status_s addresses when calling server_post_unexpected_recv(). If a failure occurs, the error code in the status structure is updated with the error value. In this case, the code would NOT segfault, because the address is actually within valid address space on the heap; however, the code would be overwriting other global structures, like the known_method_table, etc. To correct this problem, I use a local job_status_s structure inside the server_post_unexpected_recv(). Please read the comments in that function to understand why this works. SVN: Change #9272 Fixed a bug when turning off malloc redirect. Fixed bug prefenting hints being passed in to open. Added a few comments. * New Feature: added usrint and ucache code allows programs to link * directly bypassing kernel * Bug Fix: rewrote layout algorithms for RANDOM and ROUND_ROBIN which * did not perform as advertized. Still a question if metadata * selection works correctly or stuffed file allocation. * Enhancement: pvfs2-ls.c - modified the display of handles from 16 characters to 20 characters. Rev 10182 * Reworked usrint initialization to support fork/exec effectively, * particularly for shells (like bash). File descriptors were moved to * a Posix shared memory object so that they can be passed from parent * to child in a fork and from one program to the next in an exec. Fork * creates a new shm object and copies the parent's into it. Exec takes * over the previous program's object but must modify internal pointers. * program loading OFS usrint for the first time creates and initializes * a new object. File descriptor status structs can be shared from * parent to child on a fork. Shared synchronization is used to manage * access to these objects. Management of free space in the shared * objects is done in unshared user space memory using fairly simple * free lists. * * Added features to gen_locks to initialize recursive and shared mutexes * and shared conds for use in the new usrint features. Rev 10075 * Added support for glob in usrint * * Fixed several bugs that involved using an uncleared buffer that becomes * corrupted. * * Added a malloc and associated functions wrapper that uses #defines to * replace malloc with PINT-malloc(). PINT-malloc() uses glibc malloc() (and not * a client program's replacement for malloc) to do memory management. Inserts * a magic number before each allocated buffer that is check on free to prevent * a number of common malloc related bugs. Zeros newly allocarted buffer. * Zeros buffer being freed. Can be adapted for other uses as well. Includes * malloc debugging calls (currently controlled distinctly from gossip but * will be integrated into gossip). Configure options can turn off some or * all of these features. Also incAudes a set of functions clean-malloc() that * calls to the default version of malloc(). This makes pint-mem.c obsolete. * * Began migrating to a new initialization mechanism that uses a feature of * GCC __attrubute__((constructor)) that causes a routine to run before main. * This should enable us to remove the many calls to init routines currently in * the usrint code. May not work on other compilers, but there may be an * older alternative. Worst case we replace the many calls with macros. A * similar destructor attribute exists for shutdown * * Reworked implementation of stdin, stdout, stderr so that we can use them * effectively with freopen and OrangeFS files. Reworked freopen to support * this. Can still be disabled with configure. * * Made some changes to handling of locks in streams. orangefs-2.8.7 (Subversion Tag v2.8.7) Trunk * New Feature: added human readable times to pvfs2-db-display. Rev 9366. * New Feature: when setting the mode (admin or normal) of a filesystem * using pvfs2-set-mode, you can now do so for individual servers. Prior * to this change, the mode was set for all the servers within a particular * filesystem. * New Feature: users can now specify at configure time how they want the * kernel module to handle the file pointer when an error occurs. The * default behavior is to leave the file pointer alone, i.e., the file * pointer will always represent the position of the last byte written, * even though the user's entire request is not totally satisfied. In * this case, a the actual number of bytes written is returned to the user. * The new feature will change this default behavior such that the file pointer * is repositioned to the byte just prior to the user's write request, a * -1 is returned to the user's call, and errno is appropriately set. To * enable this feature, add --enable-reset-file-pos on the configure * command line when your OrangeFS system is configured. * Bug fix: modified pvfs2_readdir, which is executed by the kernel module * on behalf of an "ls", to properly retry or quit when pvfs2-client-core * aborts/restarts or when pvfs2-client terminates. Prior to these * changes, pvfs2_readdir could possibly corrupt another process's "ls" * command by releasing access to a shared memory buffer index that may * be legitimately in use. * Bug fix: modified the kernel code to re-post data to the shared memory * system (for I/O requests) whenever pvfs2-client-core aborts. Because the * data was not re-posted prior to these changes, the resulting file contained * corrupted data. * New Feature: added hint support to pvfs_open() to allow user to * provide various things like the number of servers to use, the * distribution to use, the layout to use, as well as tags that can be * used to trace requests. * New Feature: added new layout LOCAL which puts a file on the server on * the same node as the client, if there is one. Otherwise defaults to * ROUND_ROBIN * Bug Fix: repaired a bug in the layout processing * Bug Fix: fixed some problems with void pointers and replaced a missing * function prototype * Bug Fix: Corrected a long time bug: if the number of unexpected requests defined in the config file is greater than 64, then the code inadvertently used invalid job_status_s addresses when calling server_post_unexpected_recv(). If a failure occurs, the error code in the status structure is updated with the error value. In this case, the code would NOT segfault, because the address is actually within valid address space on the heap; however, the code would be overwriting other global structures, like the known_method_table, etc. To correct this problem, I use a local job_status_s structure inside the server_post_unexpected_recv(). Please read the comments in that function to understand why this works. SVN Changeset 9273 * Enhancement: Incremented display of handles from 16 to 20 bytes in pvfs2-ls. * Bug fix: Fix invalid argument errors on chmod though pvfs2fuse * Bug fix: uninitialized variable could cause seg fault when file system * cannot be initialized. orangefs-2.8.4 * Change verisioning to no longer include dates, just increment the sub version for every release orangefs-2.8.3-20110323 * Bug fix: resolve assert in dbpf_open_cache_remove() when using tree code for file remove CVS: Orange-Branch:bligon:20110322141529 * Bug fix: PINT_dist_encode() segfault in pvfs2-server caused by the client create state machine not sending dist information on create retries. CVS: Orange-Branch:mtmoore:20110322014656 CVS: Orange-Branch:mtmoore:20110322031703 CVS: Orange-Branch:mtmoore:20110322040628 CVS: Orange-Branch:mtmoore:20110322114637 CVS: Orange-Branch:mtmoore:20110322121208 * Bug fix: fix incorrect attribute output using sys attrs CVS: Orange-Branch:mtmoore:20110301023749 * Bug fix: fix segfault in pvfs2-server if client provides too large of buffer in the list-eattr state machine. Includes additional sanity checks on extended attribute operations CVS: Orange-Branch:mtmoore:20110225222047 * Bug fix: merge dir.c fix from main branch CVS: Orange-Branch:bligon:20110126215228 * Bug fix: resolve data corruption caused by ki_pos not being updated during synchronous I/O through asynchronous path (relevant to newer kernels). CVS: Orange-Branch:mtmoore:20110123143300 * Bug fix: fix resolving symbolic links from pvfs2-* utilities CVS: Orange-Branch:mtmoore:20110114215452 * Bug fix: Fix ncache entry update when moving files by David Metheny CVS: Orange-Branch:mtmoore:20110111154332 * Bug fix: PVFS_hint_add fix by Julian Kunkel and refinement of patch CVS: Orange-Branch:mtmoore:20110105051448 CVS: Orange-Branch:bligon:20110224203131 * Bug fix: change where unlocks occur w.r.t __PVFS_JOB_THREADED__ to prevent deadlock CVS: Orange-Branch:mtmoore:20101221163004 * Bug fix: change incorrect use of union members in tree-communicate CVS: Orange-Branch:mtmoore:20101214182035 * Bug fix: initialize values in trove-migrate to resolve segfault CVS: Orange-Branch:bligon:20101213175120 * Bug fix: pvfs2-mkspace not set collection-only properly CVS: Orange-Branch:bligon:20101213174458 * Bug fix: update iteration in quickhash to use safe variant CVS: Orange-Branch:mtmoore:20101207004159 * Bug fix: null handle/fs_id sent to getattr in client create state machine CVS: Orange-Branch:bligon:20101201170021 * Added support for newer (> 2.6.18) kernels up through 2.6.34. CVS: Orange-Branch:mtmoore:20101216185529 CVS: Orange-Branch:mtmoore:20101221153412 CVS: Orange-Branch:mtmoore:20101221153514 CVS: Orange-Branch:mtmoore:20101221153709 CVS: Orange-Branch:mtmoore:20101221153806 CVS: Orange-Branch:mtmoore:20101221154012 CVS: Orange-Branch:mtmoore:20101221154240 CVS: Orange-Branch:mtmoore:20101221160327 CVS: Orange-Branch:mtmoore:20110123143015 CVS: Orange-Branch:mtmoore:20110124130323 CVS: Orange-Branch:mtmoore:20110203182026 * Added beginning of new user interface CVS: Orange-Branch:walt:20110208213934 * Cleanup: change PINT_dist_finalize to unregister and free distributions cleanly CVS: Orange-Branch:mtmoore:20101221163224 * Cleanup: delete items from timer queue list when element is freed CVS: Orange-Branch:mtmoore:20101221163041 ------------------------------------------------------------------------------- orangefs-2.8.2-20100920 * Changed unstuff state machine to handle writing to files with read-only permissions on create CVS: Orange-Branch:bligon:20100920181650 * Bug fix for kernel panics from list corruption CVS: Orange-Branch:mtmoore:20100915132022 * Trove version increase to 0.1.5 to support precreate pools of multiple types CVS: Orange-Branch:mtmoore:20100908153740 * PJMP memory leak fix CVS: Orange-Branch:bligon:20100906203050 * BMI memory leak fix CVS: Orange-Branch:bligon:20100831190531, Orange-Branch:bligon:20100831190748, Orange-Branch:bligon:20100902155433