In the “Replace in string” step you can also use RegEx to remove variable content.
Remove a pattern in string with RegEx
Tag: Regex
Extract email from website
How to extract the email of the corresponding author of a publication, like: https://doi.org/10.1039/C7CS00709D with Pentaho Data integration?
- Get the HTML of the publications via REST Step, store it in one field.
- Extract email via “Regex evaluation” step using the Regex
.*mailto:([a-zA-Z0-9._-]+@[a-zA-Z0-9._-]+\.[a-zA-Z0-9_-]+).*
with the step options:- Enable dotall mode
- Enable multiline mode
The first email appearing in the HTML will put into the filed email.
Alternatively the Online Service https://www.convertcsv.com/email-extractor.htm also provides a nice possibility to extract emails from several websites:
Regex Evaluation – Arxiv ID
You can extract a pattern as an additional field with Pentaho using the “Regex evaluation” step.
Example to extract the Arxiv-ID with this Regex: .*(\d{4}\.\d{4,5}).*
Import from several Excel-Files (XLSX)
To merge several similar Excel Files in one stream, you can read out a directory with the the wildcard:
.*\.xlsx
For large XLSX Files use the “Apache POI Streaming” engine (not “Apache POI”)
Pentaho & Apache Hop Hints 