Memory leak after each Excel import in Essential Import Utility 2.10.1

[pongsakornr]

Hi Essential developers,
I have identified a repeatable memory leak in the Essential Import Utility when executing an Excel import against a server-based Protégé repository.

Environment:

• Essential Import Utility: 2.10.1

• Tomcat 9.0.x

• OpenJDK 8

• Maximum heap: 4 GB

• Remote/server-based Protégé repository

Steps to reproduce:

• Open run_excel_import.zul.

• Select an import activity.

• Under “Execute Import”, select a server-based production environment.

• Click “Execute Import”.

• Click “Run Import” and wait for successful completion.

Repeat the import several times.
Run a full GC and compare Java heap histograms.

Observed behaviour:
Each execution retains approximately 200–400 MB. The memory remains reachable after the import succeeds and is not recovered by a full GC. Eventually, repeated imports can exhaust the configured heap.
In my test, live heap increased from approximately 1.15 GB after the first run to 1.92 GB after the third run—an increase of about 787 MB.
The growing classes are almost entirely Protégé repository structures:
RemoteClientProject: 4 -> 7
DefaultKnowledgeBase: 8 -> 14
RemoteClientFrameStore: 4 -> 7
Protege Session: 4 -> 7
framestore.Record: 2,570,560 -> 4,498,480
The largest memory increases were in HashMap$Node, HashMap backing arrays, Protégé Record, FrameID, DefaultSimpleInstance, and related knowledge-base collections.
Jython objects remained stable, and the Apache POI workbook was not the principal retained object.

[neil.walsh]

Hi pongsakornr,

Thanks for raising this, it sounds like we are holding multiple copies of the repository in memory where we don't need to. This has probably been a long standing bug that we have missed.

This is interesting timing as we are currently planning the next open source release and this includes some big changes to the Import Utility. Since we launched Essential Cloud in 2017, the Import Utility there has offered a different experience. In Cloud, we removed the ability to "execute" imports in favour of defaulting to creating downloadable DUP files that users apply manually. Whilst this might seem like a regression, functionally speaking, the benefits of repeatability - having a DUP file to reapply - have been invaluable. It is definitely something the whole team now prefer.

In our experience, the Protege integration has always been brittle, memory intensive and challenging for users to configure. With that in mind, with the next release of the Import Utility, we are planning to release the DUP only version - removing the direct Protege integration. This should ensure features from the Cloud version get into the open source version more quickly and improves maintainability for the team.

We also have some exciting improvements coming to the Open Source version soon as we look to modernise this for the future. I can't share anything specific yet but we're working on getting more features from our Cloud product to open-source where we can.

Neil