Set up distributed index using Hibernate Search and Lucene -

our application using hibernate search indexing of data. application running on 2 jboss eap 6.2 application servers load distribution , failover. need changes made on 1 machine visible on other. index central part of application , needs consistent database data. rebuilding takes long time important remains intact in case of server crash. also, index expected grow large keep of in memory.

our current solution use standard filesystem directory shared filesystem (nfs) , jgroups backend ensure 1 server writes given index @ time. works more or less, have problems index updates taking long (up 20 seconds) or failing completely. due other reasons need migrate away used file system, evaluating alternatives current setup.

one thing tried infinispan directory file cache store persistence, had problems there regarding outofmemoryerrors (see post in infinispan forums https://developer.jboss.org/thread/253732). also, performance still not acceptable in our first tests (about 3 seconds index update 2 clustered servers set on developer machine), though may due configuration issues.

i think not such uncommon requirement, couldn't find information on best practices implement it.

who has experiences similar setups? infinispan directory work you? can suggest working configuration or how proceed arrive @ one? alternatives have tried , work?

you need careful versions being used. infinispan version bundled within jboss eap not intended (i.e. tested extensively other purposes) storing lucene index.

when jboss eap 6.2 released, bundled infinispan considered go internal needs of application server, might have discovered, feature of index storage having @ least performance issues.

in recent developments of infinispan applied many improvements index storage feature, fixing bugs , getting significant performance improvements out of it. hope willing try infinispan 7.2.0.beta1 ? of these improvements being backported jboss data grid, version 6.5 make them available supported product. note feature of storing hibernate search index wasn't supported before - going new feature of jdg 6.5. modules jdg 6.5 compatible jboss eap, you'll have make sure you'll use infinispan build provided jdg , not 1 meant internal usage of eap.

performance improvements still being worked on. it's better - compared older version - won't stop working on yet if try latest bleeding edge versions of infinispan 7.2.x (another release scheduled tomorrow), i'd highly appreciate feedback keep pushing it.