java - Storing and reading large arrays -

i'm recreating level data structure 2d game. i've used large 2d byte arrays levels , therefore able keep them inside memory without problems, i'm expanding game , cannot store data inside memory. i've recreated level structure this.

code single tile:

public class tile {      public static final int size = 16;      private short id;     private short health;     private boolean solid;      ... } 

instead of storing tiles 1 single array, split large array smaller arrays - chunks:

public class chunk {      public static final int width = 16;     public static final int height = 16;      private tile[][] tiles;      private int chunkx;     private int chunky;      ...  } 

and keep chunks:

public class map {      public static final int extra_draw_width = 0;     public static final int extra_draw_height = 0;      private chunk[][] chunks;      private int width;     private int height;      ... } 

the problem facing can't figure out on how store these chunks onto disk , later on read them 1 one traverse level (i want load nearest chunks game entities). far have tried:

• store each chunk in separate file. larger worlds file count became way big, example 4096 (i have keep chunks of small size in order update least game entities possible).

• store chunks single text file, though not figure out fast way on how specific chunks need.

• i've looked fast-serialization couldn't work out on how read specific chunks file well. ran memory problems when using fast-serialization , serialization.

ideally i'd have chunks inside single file , specify ones load. there libraries or specific methods of doing this?

if can ensure every tile , every chunk has same size on disk, can map chunk directly onto position in file.

example:

seekablebytechannel channel; bytebuffer chunkbuffer;  public void open(path path) {     channel = files.newbytechannel(path, enumset.of(read, write, sparse)));     chunkbuffer = bytebuffer.allocate(chunk.size); }  public void close() {     channel.close();     chunkbuffer = null; }  public void write(chunk chunk) {     int index = chunkindex(chunk.getx(), chunk.gety());     chunkbuffer.clear();     chunk.saveinto(chunkbuffer);     chunkbuffer.flip();     channel.position(header_size + chunk.size * index);     channel.write(chunkbuffer); }  public chunk read(int x, int y) {     int index = chunkindex(x, y);     chunkbuffer.clear();     channel.position(header_size + chunk.size * index);     if (channel.read(chunkbuffer) < 0) {         /* end-of-file or chunk @ given index not written yet */         return null;     } else {         chunkbuffer.flip();         return chunk.loadfrom(chunkbuffer);     } }  /** compute linar index of chunk @ position x/y */ private int chunkindex(int x, int y) {     return y * max_chunks_x + x; } 

saving , loading chunk objects:

public class chunk {     public static final int width = 16;     public static final int height = 16;     public static final int size = width * height * tile.size;      private tile[][] tiles;      public void saveinto(bytebuffer buf) {         (int x = 0; x < width; ++x) {              (int y = 0; y < height; ++y) {                 tiles[x][y].saveinto(buf);             }         }     }      public static chunk loadfrom(bytebuffer buf) {         chunk chunk = new chunk();         (int x = 0; x < width; ++x) {              (int y = 0; y < height; ++y) {                 tiles[x][y] = tile.loadfrom(buf);             }         }     }     ... } 

saving , loading tile objects:

public class tile {     public static final int size = 16;      private short id;     private short health;     private boolean solid;      public void saveinto(bytebuffer buf) {         buf.putshort(id);         buf.putshort(health);         buf.put(solid ? 1 : 0);         ...         // make sure write same tile size!         // fill placeholder if necessary!     }      public static tile loadfrom(bytebuffer buf) {         tile tile = new tile();         tile.id = buf.getshort();         tile.health = buf.getshort();         tile.solid = buf.get() == 1;         ...     } } 

of course might add range checks , proper exception handling!