Better to choose a right set of tools. You can perform ETL for 1TB of data using one single machine and bash. I would look into map-reduce like frameworks and infrastructure. Like SQS + AWS lambda + RedShift/Snowflake
or on our own infra Kafka + map and reduce in go/python/java-Spark + Cassandra/HBase/BigTabe/Mongo/Elastic/etc..
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Better to choose a right set of tools. You can perform ETL for 1TB of data using one single machine and bash. I would look into map-reduce like frameworks and infrastructure. Like SQS + AWS lambda + RedShift/Snowflake
or on our own infra Kafka + map and reduce in go/python/java-Spark + Cassandra/HBase/BigTabe/Mongo/Elastic/etc..