Ask a Question

Why can't store a date into Microstream DB, or what's wrong?

0 votes
asked in MicroStream for Java by (590 points)

I import datarows from a mysql db-table into a microstream storage.
So far import and export works fine for string, double and Integer fields. I have just a problem by importing a date field.
My code for importing:
 

                for(final TUmsatzClassification TUmsatzClassification : allUmsatzClassif_s)
                {
                    final msTUmsatzClassification myMsTUmsatzClassification = new msTUmsatzClassification();
                    root.allUmsClassifications.add(myMsTUmsatzClassification);
                    storage.store(root.allUmsClassifications);
                   
                    z++;
                    final Integer myId = TUmsatzClassification.getId();

 // here i print out correct values just for testing purpose as you can see below
                    System.out.println("Rückgabewert TUmsatzClassification.getDatum() : "
                        + TUmsatzClassification.getDatum().toString());

                    myMsTUmsatzClassification.setId(myId);
                    myMsTUmsatzClassification.setPos(Integer.valueOf(TUmsatzClassification.getPos()));
                    ....
                    myMsTUmsatzClassification.setDatum(TUmsatzClassification.getDatum());
                    ....
                    storage.store(myMsTUmsatzClassification);

 // here i print out correct values just for testing purpose as you can see below

                    System.out.println("Rückgabewert myMsTUmsatzClassification.getDatum() : "
                        + myMsTUmsatzClassification.getDatum().toString());
                }
 

The result in console is:
Rückgabewert TUmsatzClassification.getDatum() : 2019-11-27
Rückgabewert myMsTUmsatzClassification.getDatum() : 2019-11-27
Rückgabewert TUmsatzClassification.getDatum() : 2019-11-28
Rückgabewert myMsTUmsatzClassification.getDatum() : 2019-11-28
...

So far everything looks fine.

Then I stop the application, (just to shutdown storage) and restart it.
By following code I try to print out the table content:

            for(final msTUmsatzClassification msTUmsatzClassification : root.allUmsClassifications)
            {
                z++;
                sb.append(Integer.toString(msTUmsatzClassification.getId()) + " | ");
 ...

              final String strDate = dateFormat.format(msTUmsatzClassification.getDatum());
                sb.append(strDate + " | ");
                sb.append("\r\n");
            }
 

The result contains wrong date like: 1970-01-01
The same result without converting: msTUmsatzClassification.getDatum()
The same result, when I use the RC Grid to autput the data.

Field Definitions out of class msTUmsatzClassification:
(Entity: -+ same Definition like used for MySQl Entity )

    private Date    datum;
....
    @Caption("Datum")
    @Temporal(TemporalType.DATE)
    @Column(name = "datum", nullable = false, columnDefinition = "DATE", length = 10)
    public Date getDatum()
    {
        return this.datum;
    }
    public void setDatum(final Date datum)
    {
        this.datum = datum;
    }

So, what did I wrong, or why are the output wrong?

Thanks in advance!
rgds OpaHeinz

3 Answers

0 votes
answered by (1k points)

Did you try

 storage.storeRoot();

 before 

 storage.store(myMsTUmsatzClassification) ?

commented by (590 points)
I assume, that it should not be a difference, because I store a set of different fields with different types by each loop of the  for(final TUmsatzClassification TUmsatzClassification : allUmsatzClassif_s) {...}
Every field is stored and persisted except the date field.

I searched the microstream handbook and documentation, and I could not found a description which datatypes the database supports. --> I think because microstream should be able to support each java native datatype and by able to store object graphs implicidly also type date.
+2 votes
answered by (9.7k points)
edited by

Hello,

Most likely you use the type java.sql.Date. This type needs a custom type handler to be implemented to allow persisting. The reason for this behavior is that all fields of that class are marked transient.
As workaround use the type java.util.Date in your code if possible. This type is supported by Microstream by default.

Alternative you can implement an custom type handler for java.sql.Date or set a custom Field evaluator.

Edited: removed custom PersistenceFieldEvaluator example

commented by (590 points)
Hello Harald, thank you for answer!
I didn't wrote it, but source data entity and target data entity still use import java.util.date as date.type.

But what I had not in mind, I took over by RC MySQl Entity, the temporal annotation!
@Temporal(TemporalType.DATE) wich maps as java.sql.date
I will play a little bit, also with you suggestion and will come back with the result!
commented by (9.7k points)
Here is a corrected example for an custom PersistenceFieldEvaluator


EmbeddedStorageManager storage = EmbeddedStorage
    .Foundation(workDir)
    .onConnectionFoundation(f->
        {
            f.setFieldEvaluatorPersistable((final Class<?> entityType, final Field field) ->
                    !XReflect.isTransient(field)
                    || entityType == java.sql.Date.class
                    && field.getName().equals("fastTime"));
        })
    .start();
commented by (590 points)

Hi Harald
I tried to implement your suggestion into my dbHandler Java.
But I assume by my small experience, i couldn't get it running.

	static
	{
		final EmbeddedStorageManager storageManager = Configuration.Default()
			.onConnectionFoundation(f -> {
				f.setFieldEvaluatorPersistable(
					(final Class<?> entityType, final Field field) -> !XReflect.isTransient(field)
						|| entityType == java.sql.Date.class
							&& field.getName().equals("fastTime"));
			})
			.setBaseDirectory("C:/RC_MS_WS/FinMgmtBasic/database")
			.setBackupDirectory("C:/RC_MS_WS/FinMgmtBasic/databaseBck")
			.createEmbeddedStorageFoundation()
			.setRoot(dbHandler.root)
			.createEmbeddedStorageManager(dbHandler.root)
			.start();

		// Lazy References timeout after 10 minutes without access
		LazyReferenceManager.set(LazyReferenceManager.New(
			Duration.ofMinutes(10).toMillis()).start());

	}

The simple implementing in my storage manager configuration was not possible. I got an error onConnectionFoundation(.. that it is undefined for configuration.
Then I tried to change my configuration to use your example.
But with this I got a deprecated for .Foundation(..
Out of the Microstream documentation I was not able to figure out a combined solution.
It would be nice if you could give me again a little push to the right direction smiley
Thanks again!

commented by (9.7k points)

Hello,

You can solve the this by reordering:

.onConnectionFoundation(..) is part of the EmbeddedStorageFoundation class created by
.createEmbeddedStorageFoundation().

Additionally the lines

    .setRoot(dbHandler.root)
    and
    .createEmbeddedStorageManager(dbHandler.root)

may be replaced by .start(dbHandler.root)

final EmbeddedStorageManager storageManager = Configuration.Default()			
    .setBaseDirectory("C:/RC_MS_WS/FinMgmtBasic/database")
    .setBackupDirectory("C:/RC_MS_WS/FinMgmtBasic/databaseBck")
    .createEmbeddedStorageFoundation()
        .onConnectionFoundation(f -> {
            f.setFieldEvaluatorPersistable(
                (final Class<?> entityType, final Field field) -> 
                    !XReflect.isTransient(field)
                    || entityType == java.sql.Date.class
                    && field.getName().equals("fastTime"));
            })
    .start(dbHandler.root);

 

commented by (590 points)

Hi Harald
I tried it shortly, but it still did not run.
on the line:

f.setFieldEvaluatorPersistable

I got the error:

The method setFieldEvaluatorPersistable(PersistenceFieldEvaluator) in the type 
 PersistenceFoundation<Binary,capture#2-of ?> is not applicable for the arguments ((final Class<?> entityType, 
 final Field field) -> {})

on the line:

(final Class<?> entityType

I got the errors:

Multiple markers at this line
	- Field cannot be resolved to a type
	- This lambda expression refers to the missing type Field

I'll analyze it tomorrow, walk through the documentation and will come back -> First I have to understand this nested lambda blush
Thanks again!

commented by (590 points)

Ok, now I am back smiley
The problem with the errors is still fixed. But I am sorry, I have still now clue why?
I opend the code, analysed it but changed nothing.
I did a maven update - without changing the situation, searched things in internet and in a java book. after around an hour I came back to RapidClipseX and the errors are gone. surprise
I accept it and I was happy laugh

After this your code worked fine, but unfortunately it did not solve my date problem ...
I had a look into the PersistenceTypeDictonary.ptd file and found that my respective field has still java.util.Date type.

0000000000001000053 com.rieders.finmgmtbasic.dbmodel.msTUmsatzClassification{
java.lang.Integer com.rieders.finmgmtbasic.dbmodel.msTUmsatzClassification#id        ,
java.lang.Integer com.rieders.finmgmtbasic.dbmodel.msTUmsatzClassification#UId       ,
java.lang.Integer com.rieders.finmgmtbasic.dbmodel.msTUmsatzClassification#pos       ,
java.lang.Integer com.rieders.finmgmtbasic.dbmodel.msTUmsatzClassification#l2Id      ,
java.lang.Integer com.rieders.finmgmtbasic.dbmodel.msTUmsatzClassification#l3Id      ,
java.lang.Integer com.rieders.finmgmtbasic.dbmodel.msTUmsatzClassification#l1Id      ,
java.lang.String  com.rieders.finmgmtbasic.dbmodel.msTUmsatzClassification#rule      ,
//-----------------
java.util.Date    com.rieders.finmgmtbasic.dbmodel.msTUmsatzClassification#datum     ,
//-----------------
java.lang.Integer com.rieders.finmgmtbasic.dbmodel.msTUmsatzClassification#fixedCosts,
double            com.rieders.finmgmtbasic.dbmodel.msTUmsatzClassification#sbetrag   ,
double            com.rieders.finmgmtbasic.dbmodel.msTUmsatzClassification#dbetrag   ,
}

So, any other idea?

 

0 votes
answered by (590 points)
Due to the situation, that I found no solution for storing the date in my microstream DB by using type java.util.date,
I changed my field type to LocalDate. This worked fine!
commented by (9.7k points)

Hello,

I’m glad to read that you found a solution for the Date issue.

I am still wondering what causes those trouble to store the java.util.Date in your case, especially because everything else works. From our side all tests are fine, we where not able to create a similar problem.

If you are interested to have a deeper look in the stored data you may have a look at https://forum.microstream.one/?qa=45/csv-export-doesn-t-work, here you can find an example how to export all storage data to an human readable format.

 

Best regards

Harald

Notes: Every question must be a separate forum post. Headline: Formulate your question shortly and precisely. Thank you!
Powered by Question2Answer
...