Journals

Conferences

Frequent Links

Open Access Articles- Top Results for Linear Tape File System

Linear Tape File System

Linear Tape File System (LTFS) refers to both the format of data recorded on magnetic tape media and the implementation of specific software that uses this data format to provide a file system interface to data stored on magnetic tape. The Linear Tape File System format is a self-describing tape format developed by IBM to address tape archive requirements. The LTFS Format specification, which was adopted by the LTO Technology Provider Companies, defines the organization of data and metadata on tape - files stored in hierarchical directory structure. Data tapes written in the LTFS Format can be used independently of any external database or storage system allowing direct access to file content data and file metadata. This format makes it possible to implement software that presents a standard file system view of the data stored in the tape media. This file system view makes accessing files stored on the LTFS formatted media similar to accessing files stored on other forms of storage media such as disk or removable flash drives.

The Linear Tape File System format is an open description of the layout of data-structures stored on a sequential-access media. These data-structures hold the file content data and associated file metadata. Data media, such as LTO data tape, written using this format can be exchanged between systems that understand the Linear Tape File System format. Software systems that understand the format can provide users with a file system view of the media. Software systems may alternatively understand the format only to the degree that allows the system to read data from the media, or produce a tape that can be accepted by other systems that implement the Linear Tape File System format.

Announcement

The Linear Tape File System format was announced on April 12, 2010, and presented at the National Association of Broadcasters (NAB) trade show.[1] This announcement was made in conjunction with announcing the public availability of the IBM Long Term File System software product that uses the LTFS Format.

History

Magnetic tape data storage has been used for over 50 years. Prior to the introduction of the Linear Tape File System format, tapes used in open systems typically did not hold file metadata in a form that is easy to access or modify independent of the file content data. Instead, the tapes often contained files stored in a trivial sequence, without any directories or even filenames. Various external databases were used to maintain file metadata (file names, timestamps, directory hierarchy) to hold this data independent of the tape media.

For example, both backup software and hierarchical storage management (HSM) software typically maintain a database separate from the tape media. This type of software often allows indexing and accessing data spread over entire tape libraries, but these external databases are generally not designed for interoperability. Tapes in the system may or may not contain an index of their content. The index format depends on the software used to write the tape and is generally not interoperable with other systems.

In Unix-like systems, the tar format is an interoperable standard designed to allow encapsulation of multiple files for tape storage. Tar includes the files content and file-based metadata in a per-file header structure but does not maintain a central index of files. Tar allows one to create self-describing tapes, but does not provide a filesystem interface or characteristics. The tar format is not well-suited to allow modification of file metadata independent of modifying file content data.

The LTFS technology was first implemented by IBM as a prototype running on Linux and Mac OS X during 2008/2009. This prototype was demonstrated at NAB 2009. Based on feedback from this initial demonstration and experience within IBM the filesystem was overhauled in preparation for release as a product. The LTFS development team worked with the vendors of LTO tape products (HP and Quantum) to build support and understanding of the LTFS format and filesystem implementation leading up to the public release. The LTFS Format Specification and filesystem implementation were released on April 12, 2010 and enjoyed the support of IBM, HP, Quantum, and the LTO Consortium.

Format specification

The early versions of LTFS released in 2010 were based on the on-media format described in the "LTFS Format Specification v1.0" document. In March 2011 IBM released an updated document "LTFS Format Specification v2.0.0". The updated specification improved the text to remove ambiguity and clarify the intent of the specification. Additionally, the updated specification extends the LTFS Format to:

  • support sparse files,
  • support persistent file identifiers,
  • define virtual extended attributes for filesystem metadata and control, and
  • define minimum and recommended blocksize values for LTFS Volumes. (Primarily for compatibility across various HBA hardware implementations.)

Version 2.0.0 of the LTFS Format Specification defines rules for how the Format Specification version number may change in future and how compatibility is maintained across varying implementations of the specification. In particular, all implementations compliant with version 2.0.0 (and later versions) must correctly read media that was compliant with any prior versions of the LTFS Format Specification. All implementations compliant with version 2.0.0 (and later versions) must write media that is compliant with the "current" version of the LTFS Format Specification. Where the "current" version is the version of the LTFS Format Specification with which the implementation claims compliance. This specifically means that media modified by an implementation compliant with version 2.0.0 of the format specification will implicitly update the media to be compliant with LTFS Format Specification version 2.0.0. (This update only impacts the LTFS Index written to the media by the updated implementation.)

Implementations in compliance with version 2.0.0 (and later versions) of the LTFS Format Specification will correctly read all media generated using prior versions of the LTFS Format Specification.

Support for the LTFS Format Specification version 2.0.0 is included in the LTFS-SDE implementation of the Linear Tape File System version 1.2.0.

LTFS Format Specification History
Format Specification Version Specification Published Conforming Software
LTFS Format Specification v1.0 April 2010 IBM Long Term File System (LTFS) v1.0.0, v1.0.1
HP Linear Tape File System (LTFS) v1.0.0, v1.1.0
LTFS Format Specification v2.0.0 March 2011 IBM Linear Tape File System - Single Drive Edition (LTFS-SDE) v1.2.0,
Oracle StorageTek Linear Tape File System, Open Edition v1.0.0
LTFS Format Specification v2.1 October 2012 IBM: Linear Tape File System (LTFS) Format Specification

SNIA Technical Work Group

In August 2012, SNIA announced[2] that it was forming a TWG (Technical Work Group) to continue technical development of the specification. LTFS Format Specification v 2.1 is the baseline for the technical work and standards accreditation process; SNIA LTFS TWG members include HP, IBM, Oracle and Quantum.

Implementations

IBM Linear Tape File System - Single Drive Edition

IBM Linear Tape File System - Single Drive Edition
Developer(s) IBM
Initial release April 2010
Stable release 1.3.0.2 (4101) / 6 December 2013; 6 years ago (2013-12-06)
Development status Active
Written in C
Operating system Linux, Mac OS X, Microsoft Windows
Platform x64, Intel x86 - 32-bit
Available in English
Type Storage software
License GNU Lesser General Public License
Website ibm.com/systems/storage/tape/ltfs/

The IBM Linear Tape File System - Single Drive Edition product is software developed by IBM to implement a tape-based file system using the Linear Tape File System Format. This software implements the LTFS Format and allows tapes to be formatted as an LTFS Volume. These LTFS Volumes may then be mounted using the Linear Tape File System software allowing users and applications direct access to files and directories stored on the tape, including drag-and-drop of files.

Using the Linear Tape File System, files can be created on tape and accessed similar to the process of creating and accessing files on an external hard drive or a USB flash drive. Applications, such as file browsers, image viewers and media players can directly browse and access files on tape. LTFS enables easy and simple use of tapes in desktop computers and embedded systems.

IBM released the first version of the Linear Tape File System product as open source software. This release supports the Linear Tape Open Ultrium 5 (LTO-5) drives. Every release of IBM LTFS - SDE is based on the Linear Tape File System Format Specification developed by IBM and released as an open specification.

Prior to version 1.2.0 the "IBM Linear Tape File System - Single Drive Edition" product was distributed under the product name "IBM Long Term File System".

Operating systems supported by IBM LTFS-SDE

  • Red Hat Enterprise Linux (RHEL) 5.4, 5.5 with LTFS-SDE 1.2.0 for Linux (latest release: Jun 16, 2011)
  • SUSE Linux Enterprise Server (SLES) 11 SP 1 with LTFS-SDE 1.2.0 for Linux (latest release: Jun 16, 2011)
  • Mac OS X 10.5.6 (Leopard), 10.6.7 (Snow Leopard) with LTFS-SDE 1.2.0 for Mac OS X (latest release: Jun 16, 2011)
  • Windows 7 with LTFS-SDE - Windows version 1.2.0 (latest release: Sept 2, 2011)
  • Windows Server 2008 R2 with LTFS-SDE - Windows version 1.2.2 (latest release: Sept 2, 2011)

IBM Linear Tape File System - Library Edition

IBM Linear Tape File System - Library Edition
Developer(s) IBM
Initial release June 2011
Stable release 2.0.0 / 28 March 2011; 9 years ago (2011-03-28)
Development status Active
Written in C
Operating system Linux
Platform x64, Intel x86 - 32-bit
Available in English
Type Storage software
License Proprietary commercial software
Website ibm.com/systems/storage/tape/ltfs/

The IBM Linear Tape File System - Library Edition (LTFS-LE) product is software developed by IBM to implement a tape-based file system using the Linear Tape File System Format. This software implements the LTFS Format and allows tapes to be formatted as an LTFS Volume. These LTFS Volumes may be used in a tape library along with the LTFS-LE software. The LTFS-LE software allows the user to mount the tape library allowing users and applications direct access to all files and directories stored in the LTFS Volumes. When the library is mounted each LTFS-formatted tape cartridge in the library appears as a separate folder under the filesystem mount point. The user or application can navigate into each of these folders to access the files stored on each tape. The LTFS-LE software automatically controls the tape library robotics to load and unload the necessary LTFS Volumes to provide access to the stored files.

IBM Linear Tape File System - Library Edition implements the LTFS Format Specification version 2.0.0. LTFS Volumes produced in LTFS-LE are fully compatible with the open-source LTFS-SDE software product. LTFS-LE is available for all current IBM tape library products.

Operating systems supported by IBM LTFS-LE

Oracle's StorageTek Linear Tape File System, Open Edition

Oracle's StorageTek Linear Tape File System, Open Edition
Developer(s) ORACLE
Initial release 2011
Stable release 1.2.6
Development status Active
Written in C
Operating system Linux
Platform x64, Intel x86 - 32-bit
Available in English
Type Storage software
License LGPL-2.1, BSD
Website www.oracle.com/us/products/servers-storage/storage/tape-storage/ltfs/ and oss.oracle.com/projects/ltfs/

Oracle's free open source StorageTek Linear Tape File System (LTFS), Open Edition software[3] enables customers to write files to tape in a self-describing format, much the same way files are written to disk and flash storage devices.

Oracle’s StorageTek LTFS software offering is the first solution to store 8.5TB (native capacity) on a single cartridge. StorageTek LTFS software supports Oracle’s midrange StorageTek LTO 5 and LTO 6 tape drives from HP and IBM as well as Oracle’s StorageTek T10000C and T10000D tape drives.[4]

With Oracle’s LTFS software, customers have the opportunity to archive data with the StorageTek T10000C and T10000D tape drive,[5] while still maintaining the option to exchange data on a midrange StorageTek LTO 5 or LTO 6 tape drive from HP or IBM.

Oracle's StorageTek Linear Tape File System, Library Edition

Oracle’s StorageTek LTFS-LE software offering supports the StorageTek SL8500 Modular Library System, the StorageTek SL3000 Modular Library System, and the StorageTek SL150 Modular Tape.[6]

HP Linear Tape File System

HP Linear Tape File System
Developer(s) HP
Stable release 1.1.0 / 29 November 2010; 9 years ago (2010-11-29)
Development status Active
Written in C
Operating system Linux, Mac OS X
Platform x64, Intel x86 - 32-bit
License GNU Lesser General Public License
Website www.hp.com/go/ltfs

The HP Linear Tape File System (HP LTFS) is a free open source software application which provides a self-describing tape format based on the LTFS specification from the LTO Technology Provider Companies.

HP LTFS provides users with the ability to access files on an LTO-5 tape using the same commands and operations as that of a disk drive. With HP LTFS, users can share content across heterogeneous systems without the need for separate application software to read or write data to tape. Because HP LTFS is an open format, users can be sure that their data is accessible without the need for any proprietary software, making it particularly suited for long-term data archive.

Operating systems supported by HP LTFS

Quantum Linear Tape File System

Quantum LTFS
Developer(s) Quantum Corporation
Operating system Windows, Linux, Mac OS X
License GNU Lesser General Public License
Website www.quantum.com/ltfs

By Quantum Corporation. Windows, Linux and Mac OS X are supported. All Quantum Scalar Libraries with LTO5 / LTO6 drives are supported. The Scalar LTFS Appliance is a file system that presents a Quantum tape library as an NAS share. This appliance makes files viewable as if they resided on your local disk and allows users to drag and drop files directly to and from a tape cartridge. This appliance provides large file portability so that you can quickly and easily retrieve large files, whether they are active or archived, from your library, exchange files with other organizations (inside and outside of your company) for collaborative and sequential work-flow applications, and drag and drop files to and from tape.

LTFS compatible products

Enterprise Drives

  • IBM TS1140 Tape Drive
IBM 4 TB Native Capacity Tape Drive [7]
  • Oracle StorageTek T10000C Tape Drive
Oracle 5 TB Native Capacity Tape Drive [8]

LTO drives

  • HP StorageWorks LTO-5 Ultrium 3000
HP LTO5 external half height tape drive, 6 Gbit/s SAS
  • HP StorageWorks LTO-5 Ultrium 3280
HP LTO5 full height tape drive, 6 Gbit/s SAS
  • IBM System Storage TS2250 Tape Drive
IBM LTO5 external stand-alone half height tape drive, 6 Gbit/s SAS interface[9]
  • IBM System Storage TS2350 Tape Drive
IBM LTO5 external stand-alone full height tape drive, 6 Gbit/s SAS interface[10]
  • IBM LTO5 FH FC Tape Drive
IBM LTO5 internal full height tape drive, 8 Gbit/s Fiber Channel interface[11]
  • IBM LTO5 FH SAS Tape Drive
IBM LTO5 internal full height tape drive, 6 Gbit/s SAS interface[11]
  • IBM LTO5 HH FC Tape Drive
IBM LTO5 internal half height tape drive, 8 Gbit/s Fiber Channel interface[11]
  • IBM LTO5 HH SAS Tape Drive
IBM LTO5 internal half height tape drive, 6 Gbit/s SAS interface[11]
  • Quantum LTO5 HH SAS Tape Drive
Quantum LTO5 external half height tape drive, 6 Gbit/s SAS interface[12]
  • Quantum LTO5 FH SAS Tape Drive
Quantum LTO5 internal half height tape drive, 6 Gbit/s SAS interface[12]

Appliances

  • 1 Beyond Wrangler LTO-5 Offload Stations
The 1 Beyond[13] Wrangler LTO-5 Offload Stations is an appliance to store broadcast contend directly on tape via LTFS.
  • Cache-A’s LTO-5 based products
Cache-A Corporation[14] demonstrated one of the early integrations of the LTFS format on LTO Gen5 using IBM's LTFS codebase[15] at NAB 2010. Cache-A is continuing this development with the HP codebase[16] to provide LTFS-based archive appliances that provide access to this technology without requiring any client-side software.
  • Crossroads StrongBox LTFS tape-based archive
Crossroads[17] announced on April 11th, 2011,[18] to launch in a new tape-based archive product that leverages the Linear Tape File System (LTFS) technology. This solution will serve as an online archive for tier3(Tertiary storage) data, presenting itself as disk storage, but incorporating both disk for fast file storage and retrieval, and physical LTO tape for cost-effective, long-term capacity storage.

Products available: StrongBOX T1 and StrongBOX T3.

  • FOR-A LTR-110HS/120HS Video Archive Recorder
The FOR-A[19] LTR-100HS digital video recorder device is using IBM LTFS formatted tape cartridges. It was introduced at the NAB 2010 and won the IBC 2010 Pick Hit Awards from Broadcast Engineering. Its successor is the FOR-A LTR-120HS [AVC-Intra/DVCPRO Model], which supports additional MXF formats.

ISVs (Independent Software Vendors) supporting LTFS

  • BRU PE
TOLIS Group's[20] BRU Producer's Edition version 2.3.0 provides native LTFS (Linear Tape Filesystem Support) for OS X users.[21]
  • FlashNet
SGL[22] (Software Generation Limited) a UK-based broadcast archive software solutions company, was the first Independent Software Vendor (ISV), which announced[23] their support of LTFS with their FlashNet product.
  • Masstech Group Inc.
Masstech Group Inc.,[24] a Toronto, Canada-based developer of media asset management, archiving, workflow automation and transcoding solutions for the media and entertainment industry, announced LTFS support across the company's solution portfolio in 2012.[25]
  • PreRollPost
PreRollPost[26] can be used with any LTFS compliant tape drive within Macintosh OSX 10.7 and 10.8. Backup, scheduling, and retrieving are all possible with PreRollPost. No formatting, software, nor other hardware is required. A single click is required to install everything for LTFS. Backup can be used with any files ie NLE project files or entire hard drives. PreRollPost also supports backing up camera originals with an index saved on the local hard drive or LTO tape.
  • QStar Archive Manager with LTFS Volume Spanning
QStar[27] announced on 19th July 2012,[28] QStar Unveils Industry’s First Software-Based LTFS Volume-Spanning and File-Spanning Capability for Tape Libraries. With the use of QStar’s LTFS volume-spanning technology, all media within the tape library can be seen as one or more ever-growing network share(s). New LTFS media is automatically added to the set as the previous media becomes full. This approach eliminates the constant stopping and manual reconfiguration required by standard LTFS methodologies as a tape reaches capacity.
  • XenData
XenData’s entire digital video archive product range supports LTFS,[29] offering a number of advantages compared to other LTFS solutions.[30]
  • YoYottaID
YoYottaID production and post workflow software for archiving and backup to LTFS tapes created by YoYotta.[31]

LTFS projects

Industry recognition

  • IBM LTFS technology received a Pick Hit Award from Broadcast Engineering at NAB 2011.[34]
  • IBM and FOX Networks received an Engineering Emmy Award in 2011 for a project that uses LTFS to store, exchange, and archive video content.[35]
  • IBM received the 2011 Hollywood Post-Alliance (HPA) Engineering Excellence Award.[36]

References

  1. ^ NAB-2010
  2. ^ "SNIA Announces Linear Tape File System Technical Work Group"
  3. ^ Oracle StorageTek Linear Tape File System, Open Edition Webpage
  4. ^ Oracle StorageTek T10000C Tape Drive Press Release
  5. ^ Oracle Introduces StorageTek T10000D Tape Drive
  6. ^ Oracle StorageTek LTFS, Library Edition Press Release
  7. ^ "IBM System Storage TS1140 Tape Drive Model E07 delivers higher performance, reliability, and capacity". IBM. Retrieved 4 October 2012. 
  8. ^ Oracle StorageTek T10000C Tape Drive Webpage
  9. ^ IBM TS2250 Webpage
  10. ^ IBM TS2350 Webpage
  11. ^ a b c d IBM Ultrium tape drives, libraries, and autoloaders
  12. ^ a b Quantum LTO-5 Webpage
  13. ^ 1 Beyond - Homepage
  14. ^ Cache-A - Homepage
  15. ^ IBM LFTS Linux Codebase
  16. ^ HP LTFS Linux Codebase
  17. ^ Crossroads - Homepage
  18. ^ Crossroads - Press Release, Austin, TX, April 11, 2011 : Plans to Launch Archive Product that Ushers in a New Era for Tape
  19. ^ FOR-A - Homepage
  20. ^ Tolisgroup - Homepage
  21. ^ TOLIS Group - Press Release, Las Vegas, NV — April 11, 2011: BRU PRODUCER'S EDITION (BRU PE) SUPPORTS LTFS
  22. ^ SGL - Homepage
  23. ^ SGL - Press Release, Fareham, UK - February 9, 2011: SGL announces LTFS implementation transforming digital media workflows
  24. ^ Masstech Group Inc.-Homepage
  25. ^ Masstech provides archive support for IBM’s groundbreaking Linear Tape File System Press Release
  26. ^ Imagine Products Inc.-Homepage
  27. ^ QStar - Homepage
  28. ^ QStar - Press Release, Navarre, Florida:
  29. ^ XenData - Products
  30. ^ XenData First to Launch LTO Video Archives with Multiple Format File System Support Including LTFS and tar
  31. ^ YoYotta - Homepage
  32. ^ Thought Equity Motion - Homepage
  33. ^ Thought Equity Motion - Press Release: Film Archive Digitization and Preservation Project for EYE Film Institute Netherlands
  34. ^ NAB 2011 Pick Hit Announcement
  35. ^ IBM Almaden Blog
  36. ^ HPA Award Announcement

External links