The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here
"How to process thousands of CSV files (data files) in batch mode?"
Learner III
There are about 3000 CSV files which are source data files. Each file has 7 attributes but different instances.
E.G.
Date Open-Price High-Price Low-Price Close-price Amount Volume-money
07/02/2007 0.83 0.85 0.82 0.82 1624001 13294947
07/03/2007 0.81 0.83 0.79 0.81 2837039 22465612
07/04/2007 0.81 0.84 0.8 0.82 2587505 21029224
07/05/2007 0.81 0.85 0.79 0.81 2178333 17773198
07/06/2007 0.79 0.84 0.79 0.84 2496353 20309706
07/09/2007 0.83 0.87 0.83 0.86 2556154 21822494
07/10/2007 0.86 0.86 0.82 0.82 2288000 18957010
07/11/2007 0.81 0.85 0.79 0.84 1541856 12688349
07/12/2007 0.84 0.88 0.83 0.88 3545073 30796872
07/13/2007 0.88 0.9 0.85 0.86 2222693 19366442
07/16/2007 0.86 0.88 0.84 0.85 1050127 8936699
I put all CSV files under one file folder.The files are named XXXXXXYY.csv E.G.:
000010SZ.csv
600020SH.csv
000035SZ.csv
...
1. I want to create a process to select every two CSV files to join the atrributes into one Example Set.
2. Calculate correlaction of open-price, open-price_from_ES
3. Write result to a text file.
How to make a batch to make a selection of two data files from thousands source data files and loop all the files to calculate the correlation?
Thanks in advance.
Gerald
E.G.
Date Open-Price High-Price Low-Price Close-price Amount Volume-money
07/02/2007 0.83 0.85 0.82 0.82 1624001 13294947
07/03/2007 0.81 0.83 0.79 0.81 2837039 22465612
07/04/2007 0.81 0.84 0.8 0.82 2587505 21029224
07/05/2007 0.81 0.85 0.79 0.81 2178333 17773198
07/06/2007 0.79 0.84 0.79 0.84 2496353 20309706
07/09/2007 0.83 0.87 0.83 0.86 2556154 21822494
07/10/2007 0.86 0.86 0.82 0.82 2288000 18957010
07/11/2007 0.81 0.85 0.79 0.84 1541856 12688349
07/12/2007 0.84 0.88 0.83 0.88 3545073 30796872
07/13/2007 0.88 0.9 0.85 0.86 2222693 19366442
07/16/2007 0.86 0.88 0.84 0.85 1050127 8936699
I put all CSV files under one file folder.The files are named XXXXXXYY.csv E.G.:
000010SZ.csv
600020SH.csv
000035SZ.csv
...
1. I want to create a process to select every two CSV files to join the atrributes into one Example Set.
2. Calculate correlaction of open-price, open-price_from_ES
3. Write result to a text file.
How to make a batch to make a selection of two data files from thousands source data files and loop all the files to calculate the correlation?
Thanks in advance.
Gerald
Tagged:
0
