
Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community for help and clear up your study doubts
Discover the best universities in your country according to Docsity users
Free resources
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
The configuration of apache flume for collecting data from twitter using various components such as sources, channels, and sinks. The twitter source is set up with specific keywords and authentication details, while the memory channel and hdfs sink are configured for data storage.
Typology: Study Guides, Projects, Research
1 / 1
This page cannot be seen from the preview
Don't miss anything!
agent.sources = twitter-source agent.sinks = hdfs-sink agent.channels = memory-channel
agent.channels.memory-channel.type = memory agent.channels.memory-channel.capacity = 1000 agent.channels.memory-channel.transactionCapacity = 100
agent.sources.twitter-source.type = com.hirw.twittersource.TwitterSource agent.sources.twitter-source.consumerKey = zyfFBK13ScCg8YjUhea3g agent.sources.twitter-source.consumerSecret = fSgkxQuoGIZz7lfqBu167KZy3dxl3OdXMhdTvk91Q agent.sources.twitter-source.accessToken = 2288102790-8BwzXU0mTc9nKZEuyxRiel9UulOlmTWXFfUWXia agent.sources.twitter-source.accessTokenSecret = ptLkj2IgH2RfXTLxdjyRUzTksIRtys2CTM3YPmIQfcv5o agent.sources.twitter-source.keywords = food,cuisine,restaurant,fastfood,sushi,service,quality,oil,garlic,healthy,organic,fresh,ingredien ts,waiter,server,host,reservations,taste,chef,burger,cooked,wine,beer,ambience,taco,buritto,sal sa,fries,chicken,meat,fish,italian,pasta,gourmet,sauce agent.sources.twitter-source.channels = memory-channel
agent.sinks.hdfs-sink.type = hdfs agent.sinks.hdfs-sink.hdfs.path = flume/tweets/%Y/%m/%d/%H agent.sinks.hdfs-sink.hdfs.fileType = DataStream agent.sinks.hdfs-sink.hdfs.writeFormat = Text agent.sinks.hdfs-sink.hdfs.rollCount = 500 agent.sinks.hdfs-sink.channel = memory-channel