The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

2 Database mySQL - query depend each other

sqlusersqluser Member Posts: 3 Contributor I
edited June 2019 in Help
Hello,

i have 2 mySQL Databases. First one have event entries with a date. Second database have many entries
25.567.157 rows with Value and Date.

I'd like classify the second Database depend on event entries of first database.

For example: event date is 12.08.2009 14:00:00 (DD.MM.YYYY HH:MM:SS) in first database
then i would analyse database one for this one day

at the moment i do this manually entry in first database query

SELECT `date`, `value`
FROM `myvaluelist`
WHERE `date` between 20090812000000 and 20090812235959

i won't read the whole database one with 1GB data - i only would query entries which are as event in database 2

my thought is to get from database 2 all dates from events and export this to a file - it ist possibly to transform this
automatically in a WHERE clausel and write to file - so i can set for database one the query by file

or exists an other method? loading complete database one with select * from myvaluelist occurs this
"Out of Memory - Java Heap size error"

database two have 5.215 entries with events

Thank you for help!
Tagged:

Answers

  • landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,531 Unicorn
    Hi,
    this is a very common preprocessing task we are facing and it is completely solvable using RapidMiner.
    The problem is, that I cannot give all the knowledge in detail, it would simply take far too long. This usually is taught in a two day seminar.

    The bottom line is, that you can load the complete first data base and transform it's values, so that you can use them later on for including them into the query. After you did this, you can loop over each single example and extract the values for the query into a macro. After doing this, you can make a query and include the macros into the query string. During execution they will be replaced by the values stored into the macros. This way you can load your data in chunks for each single event data.
    You will need the Loop Examples and Extract Data operator for this.

    I hope this helps. If you need more information or doesn't have the time to find it out yourself, you might book a seminar or webinar from our shop.

    Greetings,
      Sebastian
  • sqlusersqluser Member Posts: 3 Contributor I
    I have to create macros an use them in the query string. I will try to create this preprocessing task.
    Thank you for the information. I hope i get running it.
  • sqlusersqluser Member Posts: 3 Contributor I
    I did'nt get running it.

    Descripes the Whitpaper "How to Extend RapidMiner 5.0" Preis: €40.00  this step, how to set up the preprocessing task with sql databases?
    Notify i'm student and i have not the pocket money for booking a webinar or seminar.
  • landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,531 Unicorn
    Hi,
    unfortunately it does not. It describes how to write own code to extend RapidMiner's functionality. But what you need is already covered by RapidMiner. If you search the forum for macros, you will find several examples how to use them. I bet you can comprehend from them how it works out for you.

    Greetings,
      Sebastian
Sign In or Register to comment.