The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here
"How to split a tokenized document into several documents?"
Contributor I
Hello,
We download a page from the web with the module "Get Page" and split the relevant data with "Extract Content", later we want to tokenize each extracted div as a single document.
Is there a possibly to do this?
With "Cut Document" we need regular expressions but every div has a different content (movie reviews) and the HTML tags are also removed.
So we are wondering how to define a splitting point for different reviews.
Thanks in advance
We download a page from the web with the module "Get Page" and split the relevant data with "Extract Content", later we want to tokenize each extracted div as a single document.
Is there a possibly to do this?
With "Cut Document" we need regular expressions but every div has a different content (movie reviews) and the HTML tags are also removed.
So we are wondering how to define a splitting point for different reviews.
Thanks in advance
Tagged:
0
