Class PagedChannelRandomAccessSource

  • All Implemented Interfaces:
    RandomAccessSource

    class PagedChannelRandomAccessSource
    extends GroupedRandomAccessSource
    implements RandomAccessSource
    A RandomAccessSource that is based on an underlying FileChannel. The channel is mapped into memory using a paging scheme to allow for efficient reads of very large files. As an implementation detail, we use GroupedRandomAccessSource functionality, but override to make determination of the underlying mapped page more efficient - and to close each page as another is opened
    Since:
    5.3.5
    • Field Detail

      • DEFAULT_MAX_OPEN_BUFFERS

        public static final int DEFAULT_MAX_OPEN_BUFFERS
        See Also:
        Constant Field Values
      • bufferSize

        private final int bufferSize
        The size of each of the buffers to use when mapping files into memory. This must be greater than 0 and less than Integer.MAX_VALUE
      • channel

        private final java.nio.channels.FileChannel channel
        The channel this source is based on
    • Constructor Detail

      • PagedChannelRandomAccessSource

        public PagedChannelRandomAccessSource​(java.nio.channels.FileChannel channel)
                                       throws java.io.IOException
        Constructs a new PagedChannelRandomAccessSource based on the specified FileChannel, with a default buffer configuration. The default buffer configuration is currently 2^26 total paged bytes, spread across a maximum of 16 active buffers. This arrangement resulted in a 24% speed improvement over the single buffer case in parametric tests extracting text from a 2.3 GB file.
        Parameters:
        channel - the channel to use as the backing store
        Throws:
        java.io.IOException - if the channel cannot be opened or mapped
      • PagedChannelRandomAccessSource

        public PagedChannelRandomAccessSource​(java.nio.channels.FileChannel channel,
                                              int totalBufferSize,
                                              int maxOpenBuffers)
                                       throws java.io.IOException
        Constructs a new PagedChannelRandomAccessSource based on the specified FileChannel, with a specific buffer size
        Parameters:
        channel - the channel to use as the backing store
        bufferSize - the size of the buffers to use
        Throws:
        java.io.IOException - if the channel cannot be opened or mapped
    • Method Detail

      • buildSources

        private static RandomAccessSource[] buildSources​(java.nio.channels.FileChannel channel,
                                                         int bufferSize)
                                                  throws java.io.IOException
        Constructs a set of MappedChannelRandomAccessSources for each page (of size bufferSize) of the underlying channel
        Parameters:
        channel - the underlying channel
        bufferSize - the size of each page (the last page may be shorter)
        Returns:
        a list of sources that represent the pages of the channel
        Throws:
        java.io.IOException - if IO fails for any reason
      • getStartingSourceIndex

        protected int getStartingSourceIndex​(long offset)
        Description copied from class: GroupedRandomAccessSource
        For a given offset, return the index of the source that contains the specified offset. This is an optimization feature to help optimize the access of the correct source without having to iterate through every single source each time. It is safe to always return 0, in which case the full set of sources will be searched. Subclasses should override this method if they are able to compute the source index more efficiently (for example FileChannelRandomAccessSource takes advantage of fixed size page buffers to compute the index)
        Overrides:
        getStartingSourceIndex in class GroupedRandomAccessSource
        Parameters:
        offset - the offset
        Returns:
        the index of the input source that contains the specified offset, or 0 if unknown
      • sourceReleased

        protected void sourceReleased​(RandomAccessSource source)
                               throws java.io.IOException
        Description copied from class: GroupedRandomAccessSource
        Called when a given source is no longer the active source. This gives subclasses the abilty to release resources, if appropriate.
        Overrides:
        sourceReleased in class GroupedRandomAccessSource
        Parameters:
        source - the source that is no longer the active source
        Throws:
        java.io.IOException - if there are any problems
      • sourceInUse

        protected void sourceInUse​(RandomAccessSource source)
                            throws java.io.IOException
        Description copied from class: GroupedRandomAccessSource
        Called when a given source is about to become the active source. This gives subclasses the abilty to retrieve resources, if appropriate.
        Overrides:
        sourceInUse in class GroupedRandomAccessSource
        Parameters:
        source - the source that is about to become the active source
        Throws:
        java.io.IOException - if there are any problems