Package elki.persistent
Class OnDiskArray
- java.lang.Object
-
- elki.persistent.OnDiskArray
-
- All Implemented Interfaces:
java.lang.AutoCloseable
public class OnDiskArray extends java.lang.Object implements java.lang.AutoCloseable
On Disc Array storage for records of a given size.This can be used to implement various fixed size record-based data structures. The file format is designed to have a fixed-size header followed by the actual data.
- Since:
- 0.2
- Author:
- Erich Schubert
-
-
Field Summary
Fields Modifier and Type Field Description private java.nio.channels.FileChannel
file
Random Access File object.private java.nio.file.Path
filename
File name.private static int
HEADER_POS_SIZE
Position of file size (in records).private int
headersize
Size of the header in the file.private static int
INTERNAL_HEADER_SIZE
Size of the classes header size.private java.nio.channels.FileLock
lock
Lock for the file that will be kept while writing.protected int
magic
Magic number used to identify files.private java.nio.MappedByteBuffer
map
The memory mapped buffer.private int
numrecs
Number of records in the file.private int
recordsize
Size of the records in the file.private static long
serialVersionUID
Serial version.private boolean
writable
Writable or read-only object.
-
Constructor Summary
Constructors Constructor Description OnDiskArray(java.nio.file.Path filename, int magicseed, int extraheadersize, boolean writable)
Constructor to open an existing file.OnDiskArray(java.nio.file.Path filename, int magicseed, int extraheadersize, int recordsize, boolean writable)
Constructor to open an existing file.OnDiskArray(java.nio.file.Path filename, int magicseed, int extraheadersize, int recordsize, int initialsize)
Constructor to write a new file.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description void
close()
Explicitly close the file.void
ensureSize(int size)
Ensure that the file can fit the given number of records.java.nio.ByteBuffer
getExtraHeader()
Read the extra header data.protected int
getExtraHeaderSize()
Return the size of the extra header.java.nio.file.Path
getFilename()
Get the file name.int
getNumRecords()
Get number of records in file.java.nio.ByteBuffer
getRecordBuffer(int index)
Get a record bufferprotected int
getRecordsize()
Get the size of a single record.private long
indexToFileposition(long index)
Compute file position from index numberboolean
isWritable()
Check if the file is writable.private void
mapArray()
(Re-) map the data array.static int
mixMagic(int magic1, int magic2)
Mix two magic numbers into one, to obtain a combined magic.void
resizeFile(int newsize)
Resize file to the intended sizeprivate void
validateHeader(boolean validateRecordSize)
Validates the header and throws an IOException if the header is invalid.
-
-
-
Field Detail
-
serialVersionUID
private static final long serialVersionUID
Serial version.NOTE: Change this version whenever the file structure is changed in an incompatible way: This will modify the file magic, and thus prevent applications from reading incompatible files.
- See Also:
- Constant Field Values
-
magic
protected int magic
Magic number used to identify files.
-
headersize
private int headersize
Size of the header in the file. Note that the internal header is four integers already.
-
recordsize
private int recordsize
Size of the records in the file.
-
numrecs
private int numrecs
Number of records in the file.
-
filename
private java.nio.file.Path filename
File name.
-
file
private final java.nio.channels.FileChannel file
Random Access File object.
-
lock
private java.nio.channels.FileLock lock
Lock for the file that will be kept while writing.
-
writable
private boolean writable
Writable or read-only object.
-
map
private java.nio.MappedByteBuffer map
The memory mapped buffer.
-
INTERNAL_HEADER_SIZE
private static final int INTERNAL_HEADER_SIZE
Size of the classes header size.- See Also:
- Constant Field Values
-
HEADER_POS_SIZE
private static final int HEADER_POS_SIZE
Position of file size (in records).- See Also:
- Constant Field Values
-
-
Constructor Detail
-
OnDiskArray
public OnDiskArray(java.nio.file.Path filename, int magicseed, int extraheadersize, int recordsize, int initialsize) throws java.io.IOException
Constructor to write a new file.- Parameters:
filename
- File name to be opened.magicseed
- Magic number to derive real magic from.extraheadersize
- header size NOT including the internal headerrecordsize
- Record sizeinitialsize
- Initial file size (in records)- Throws:
java.io.IOException
- on IO errors
-
OnDiskArray
public OnDiskArray(java.nio.file.Path filename, int magicseed, int extraheadersize, int recordsize, boolean writable) throws java.io.IOException
Constructor to open an existing file. The provided record size must match the record size stored within the files header. If you don't know this size yet and/or need to access the extra header you should use the other constructor below- Parameters:
filename
- File name to be opened.magicseed
- Magic number to derive real magic from.extraheadersize
- header size NOT including the internal headerrecordsize
- Record sizewritable
- flag to open the file writable- Throws:
java.io.IOException
- on IO errors
-
OnDiskArray
public OnDiskArray(java.nio.file.Path filename, int magicseed, int extraheadersize, boolean writable) throws java.io.IOException
Constructor to open an existing file. The record size is read from the file's header and can be obtained bygetRecordsize()
- Parameters:
filename
- File name to be opened.magicseed
- Magic number to derive real magic from.extraheadersize
- header size NOT including the internal headerwritable
- flag to open the file writable- Throws:
java.io.IOException
- on IO errors
-
-
Method Detail
-
mapArray
private void mapArray() throws java.io.IOException
(Re-) map the data array.- Throws:
java.io.IOException
- on mapping error.
-
validateHeader
private void validateHeader(boolean validateRecordSize) throws java.io.IOException
Validates the header and throws an IOException if the header is invalid. If validateRecordSize is set to true the record size must match exactly the stored record size within the files header, else the record size is read from the header and used.- Parameters:
validateRecordSize
-- Throws:
java.io.IOException
-
mixMagic
public static final int mixMagic(int magic1, int magic2)
Mix two magic numbers into one, to obtain a combined magic. Note: mixMagic(a,b) != mixMagic(b,a) usually.- Parameters:
magic1
- Magic number to mix.magic2
- Magic number to mix.- Returns:
- Mixed magic number.
-
indexToFileposition
private long indexToFileposition(long index)
Compute file position from index number- Parameters:
index
- Index offset- Returns:
- file position
-
resizeFile
public void resizeFile(int newsize) throws java.io.IOException
Resize file to the intended size- Parameters:
newsize
- New file size.- Throws:
java.io.IOException
- on IO errors
-
getRecordBuffer
public java.nio.ByteBuffer getRecordBuffer(int index) throws java.io.IOException
Get a record buffer- Parameters:
index
- Record index- Returns:
- Byte buffer for the record
- Throws:
java.io.IOException
- on IO errors
-
getExtraHeaderSize
protected int getExtraHeaderSize()
Return the size of the extra header. Accessor.- Returns:
- Extra header size
-
getExtraHeader
public java.nio.ByteBuffer getExtraHeader() throws java.io.IOException
Read the extra header data.- Returns:
- additional header data
- Throws:
java.io.IOException
- on IO errors
-
getRecordsize
protected int getRecordsize()
Get the size of a single record.- Returns:
- Record size.
-
getFilename
public java.nio.file.Path getFilename()
Get the file name.- Returns:
- File name
-
isWritable
public boolean isWritable()
Check if the file is writable.- Returns:
- true if the file is writable.
-
close
public void close() throws java.io.IOException
Explicitly close the file. Note: following operations will likely cause IOExceptions.- Specified by:
close
in interfacejava.lang.AutoCloseable
- Throws:
java.io.IOException
- on IO errors
-
getNumRecords
public int getNumRecords()
Get number of records in file.- Returns:
- Number of records in the file.
-
ensureSize
public void ensureSize(int size) throws java.io.IOException
Ensure that the file can fit the given number of records.- Parameters:
size
- Size- Throws:
java.io.IOException
-
-