![Unboxing Big Data](/img/default-banner.jpg)
- 294
- 902 705
Unboxing Big Data
India
Registrace 24. 04. 2018
Unboxing Big Data channel has been created with an aim to teach Big Data tools from scratch. It lets you dive in to Big Data ecosystem which includes frameworks/tools such as Apache Hadoop, Apache Spark, Apache Flink, Apache Storm, Apache Hive, Apache Pig, Apache Flume, Apache Kafka and many more.
How to get CHATGPT in RSTUDIO IDE
Following are the steps to get CHATGPT in RSTUDIO :
install.packages("chattr")
library(chattr)
Sys.setenv("OPENAI_API_KEY" = "XXXXXXXXXXXXXXXXXXXXXX")
chattr_use("gpt35")
chattr_app(as_job = TRUE)
install.packages("chattr")
library(chattr)
Sys.setenv("OPENAI_API_KEY" = "XXXXXXXXXXXXXXXXXXXXXX")
chattr_use("gpt35")
chattr_app(as_job = TRUE)
zhlédnutí: 276
Video
Beginner's Guide to R : Easy Ways to do basic data analysis
zhlédnutí 274Před 3 měsíci
The functions covered in the video help us in easily perform basic data analysis. Hope you'll like it.
Handling Missing Values in R
zhlédnutí 285Před 3 měsíci
Handling Missing Values in R : is.na() function complete.cases() function colSums() function rowSums() function na.omit() function na.exclude() function drop_na() function na.fail() function na.pass() function replace_na() function
Apache Spark Structured Streaming - Socket Source
zhlédnutí 473Před 3 měsíci
Apache Spark Structured Streaming - Socket as a Source #apachespark NETCAT DOWNLOAD LINK : nmap.org/download.html#windows // Import Libraries import org.apache.spark.sql.SparkSession import org.apache.spark.sql.functions._ // Create Spark Session val spark = SparkSession .builder() .master("local") .appName("Socket Source") .getOrCreate() // Define host and port number to Listen. val host = "12...
Installing Apache Kafka on Windows 11 in 5 minutes
zhlédnutí 4,7KPřed 4 měsíci
Installing Apache Kafka on Windows 11 in 5 minutes
Apache Spark Tutorial : Querying structured data using Spark SQL
zhlédnutí 772Před 4 měsíci
Spark SQL is Apache Spark's module for working with structured data. Spark SQL lets you query structured data inside Spark programs.
Apache Spark: Dataframe API queries explained with examples (Part - 2)
zhlédnutí 419Před 4 měsíci
A DataFrame is a distributed collection of data organized into named columns. It is conceptually equivalent to a table in a relational database or a R/Python Dataframe
Apache Spark: Dataframe API queries explained with examples (Part - 1)
zhlédnutí 674Před 4 měsíci
A DataFrame is a distributed collection of data organized into named columns. It is conceptually equivalent to a table in a relational database or a R/Python Dataframe
Difference between groupByKey() and reduceByKey() in Spark RDD API
zhlédnutí 1KPřed 5 měsíci
In Apache Spark, both groupByKey and reduceByKey are transformations that can be used to process and manipulate key-value pair RDDs. However, they differ in their functionality and performance characteristics. In this video, detailed explanation of the differences between groupByKey and reduceByKey is shown.
Difference between map and flatMap in Spark RDD API
zhlédnutí 718Před 5 měsíci
Map - Return a new RDD by applying a function to each element of this RDD FlatMap - Return a new RDD by first applying a function to all elements of this RDD, and then flattening the results
Apache Spark RDD Actions with examples (Using Scala IDE)
zhlédnutí 646Před 5 měsíci
#apachespark #rddactions Apache Spark RDD Actions collect() reduce() min() max() mean() stdev() sum() saveAsTextFile() countByKey() Apache Spark Transformations : czcams.com/video/JKQExgRfpnM/video.html Running your first spark program :czcams.com/video/1kboBh3VFr4/video.html
Understanding Apache Spark Architecture
zhlédnutí 728Před 5 měsíci
Understanding Apache Spark architecture and its important components : Driver Program Spark Context Cluster Manager Tasks Executors Workers Spark Session
RDD Transformation in Apache Spark (With Example)
zhlédnutí 939Před 5 měsíci
RDD Transformations are Spark operations when executed on RDD, it results in a single or multiple new RDD’s. RDD are immutable in nature, transformations always create new RDD without updating an existing one hence, this creates an RDD lineage. RDD Transformation is of 2 types : Narrow Transformation and Wide Transformation
Apache Spark Tutorial: Running your first spark program in Scala using Eclipse IDE
zhlédnutí 1,2KPřed 6 měsíci
This video is for beginners who just started learning Apache Spark concepts. This video explains how to run first Apache Spark program in Scala using Eclipse IDE.
Running your first scala program in Scala IDE (Eclipse)
zhlédnutí 763Před 6 měsíci
Every scala program consist of 3 parts : object, main method and statement/expression. Download Link of Scala IDE for Eclipse : scala-ide.org/download/sdk.html
Practical Apache Spark GraphX in 10 minutes
zhlédnutí 1,5KPřed 6 měsíci
Practical Apache Spark GraphX in 10 minutes
Installation of Apache Spark on Windows 11 (in 5 minutes)
zhlédnutí 12KPřed 7 měsíci
Installation of Apache Spark on Windows 11 (in 5 minutes)
SQL Tutorial #31 Using EXISTS operator with subquery in SQL (With Example)
zhlédnutí 23Před 7 měsíci
SQL Tutorial #31 Using EXISTS operator with subquery in SQL (With Example)
SQL Tutorial #30 Using IN operator with subquery in SQL (With Example)
zhlédnutí 34Před 7 měsíci
SQL Tutorial #30 Using IN operator with subquery in SQL (With Example)
SQL Tutorial #28 UNDERSTANDING ROW_NUMBER() WINDOW FUNCTION IN SQL (PART-4)
zhlédnutí 24Před 7 měsíci
SQL Tutorial #28 UNDERSTANDING ROW_NUMBER() WINDOW FUNCTION IN SQL (PART-4)
SQL Tutorial #27 UNDERSTANDING LEAD() AND LAG() WINDOW FUNCTIONS IN SQL (PART-3)
zhlédnutí 43Před 7 měsíci
SQL Tutorial #27 UNDERSTANDING LEAD() AND LAG() WINDOW FUNCTIONS IN SQL (PART-3)
SQL Tutorial #26 CALCULATING RUNNING TOTAL IN SQL USING WINDOWS FUNCTION (PART-2)
zhlédnutí 27Před 7 měsíci
SQL Tutorial #26 CALCULATING RUNNING TOTAL IN SQL USING WINDOWS FUNCTION (PART-2)
SQL Tutorial #25 UNDERSTANDING WINDOW FUNCTIONS IN SQL (PART-1)
zhlédnutí 48Před 7 měsíci
SQL Tutorial #25 UNDERSTANDING WINDOW FUNCTIONS IN SQL (PART-1)
SQL Tutorial #24 SELF JOIN in SQL (with demonstration)
zhlédnutí 26Před 8 měsíci
SQL Tutorial #24 SELF JOIN in SQL (with demonstration)
SQL Tutorial #23 FULL JOIN in SQL (with demonstration)
zhlédnutí 20Před 8 měsíci
SQL Tutorial #23 FULL JOIN in SQL (with demonstration)
SQL Tutorial #22 RIGHT JOIN in SQL (with demonstration)
zhlédnutí 27Před 8 měsíci
SQL Tutorial #22 RIGHT JOIN in SQL (with demonstration)
SQL Tutorial #21 LEFT JOIN in SQL (with demonstration)
zhlédnutí 40Před 8 měsíci
SQL Tutorial #21 LEFT JOIN in SQL (with demonstration)
SQL Tutorial #20 INNER JOIN in SQL (with demonstration)
zhlédnutí 27Před 8 měsíci
SQL Tutorial #20 INNER JOIN in SQL (with demonstration)
SQL Tutorial #19 Introduction to joins in SQL
zhlédnutí 36Před 8 měsíci
SQL Tutorial #19 Introduction to joins in SQL
"localhost:8088/cluster" sir it is showing This site can’t be reached
Very fast explaining this topic.
where to run spark-shell --master yarn?
i was able to setup spark-shell. But when i m doing spark-shell --master yarn it shows Exception in thread "main" org.apache.spark.SparkException: When running with master 'yarn' either HADOOP_CONF_DIR or YARN_CONF_DIR must be set in the environment.
Hi.... I'm unable to add colour please tell me how should I do
Thank you so much sir explaination is wonderful
really helpful, thanks for posting
Sir what is r=>r in the rdd2 line
gotta mad using Gpt and stackoverflow but none of the solution works by miracle I got your video many thanks for the solution.
hello ResourceManager is not showing up in jps command and cluster link is not opening unable to start hadoop. what can be the issue?
helpful
hello sir plx help out this code shoiwng this issue FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask MapReduce Jobs Launched: Stage-Stage-1: HDFS Read: 0 HDFS Write: 0 FAIL Total MapReduce CPU Time Spent: 0 msec 2024-07-03T13:28:19,626 INFO [e19f5b0e-e06e-41dc-b307-2ce4b71af778 main] org.apache.hadoop.hive.conf.HiveConf - Using the default value passed in for log id: e19f5b0e-e06e-41dc-b307-2ce4b71af778 2024-07-03T13:28:19,626 INFO [e19f5b0e-e06e-41dc-b307-2ce4b71af778 main] org.apache.hadoop.hive.ql.session.SessionState - Resetting thread name to main
when i was trying to enter printf("Number=%d", 123) i got this type of error what can i do sir "<console>:40: error: type mismatch; found : String("Number=%d") required: org.apache.spark.sql.Column printf("Number=%d", 123) ^ <console>:40: error: type mismatch; found : Int(123) required: org.apache.spark.sql.Column printf("Number=%d", 123)"
Can you give a lecture for my students in tamilnadu with good remuneration?
Can you tell me the vvvvwry simple. Commands for beginners in Hadoop environment
Vvv.super and more Informative
link for library is empty
I did all the steps but I haven't got any bin folder in c drive and by using command prompt all files are downloaded but in this kind of format: "trunk/bin/ext/version.sh" what can i do now
Very well explained! Thank you.
Can you share me Hadoop IO libraries link?
I am still getting some errors ( ERROR SparkContext: Error initializing SparkContext.). can you share your mail id
Thank you ❤
I am facing an error " the system cannot find the path specified" Please help
set path clearly in enviromental variables
hello bro, can you provide bin folder for me
your Progra~1 is the "Program Files" folder in C drive right ??
while running this ``` hive --service schematool -dbType derby -initSchema ``` command to create metastore, i got this error, ``` "Missing hadoop installation: C:\hadoop must be set" ``` Can you please guide me how to resolve this error. I followed your hadoop setup video to install hadoop.
installed hadoop in one go. thank you
hi after above actions, i have an error, what should i do scala> 24/06/07 11:17:11 WARN GarbageCollectionMetrics: To enable non-built-in garbage collector(s) List(G1 Concurrent GC), users should configure it(them) to spark.eventLog.gcMetrics.youngGenerationGarbageCollectors or spark.eventLog.gcMetrics.oldGenerationGarbageCollectors
Share screenshot on mail
@@UnboxingBigData is this resolved i also need help in this
thank you for this demonstration sir
Bro, Thank you very much for making a comprehensive video with all required files and tools with demonstration, Keep It up! ,Its fully working only jps command not executed can be fixed by setting up java path. Thank you
I am facing error to start hive
Mucho ojo si alguna de las rutas delas variables del sistema que intervienen contiene algún espacio, Spark no lo soporta. Os aviso porque a mi me ha consumido 1h hasta encontrar el error.
Very helpful video .... Nice 👍🏼😊😊
Thank you man. Your video will help a lot of people understand dplyr.
Nice Explanation
Sir jps work nhi kar raha h
What’s the error
@@UnboxingBigData sir jps type krke kuch bhi nhi hora baaki sb cheezein chlri h jaise aapne video m btaya meine aapko mail kiya h with ss
Hello , As I am a complete beginner. Could you do a video on Installing apache hbase database local machine with the prerequisites installation of hadoop and jdk for apache hbase to work process as well.
There are videos available regarding installation of hadoop and jdk. Check in hadoop playlist
24/05/11 16:23:44 WARN GarbageCollectionMetrics: To enable non-built-in garbage collector(s) List(G1 Concurrent GC), users should configure it(them) to spark.eventLog.gcMetrics.youngGenerationGarbageCollectors or spark.eventLog.gcMetrics.oldGenerationGarbageCollectors I am getting this while executing the pyspark and whenevern i start the spark. how to fix them please help me out..
Well explained
Thank you!
sir ,error was coming error: not found: value LazyList val myFirstLazyList = 1.5 #:: 2.5 #:: 3.5 #:: LazyList.empty do have to import any package????
Write Stream instead of LazyList. Check if its working
@@UnboxingBigData Yes sir it's working,thank you sir val myFirstLazyList = 1.5 #:: 2.5 #:: 3.5 #:: Stream.empty myFirstLazyList: scala.collection.immutable.Stream[Double] = Stream(1.5, ?)
💯💯
💯💯💯💯
sir u really solved our biggest problem from where to study the syllabus sir I wanna meet you may u please give me your cabin no in lpu. i wanna meet you the videos you making are helping so many students.. thanq so much sir..
Thanks. Glad to know it helped. Cabin 25-202 c4
thanks for explaining with the use of examples sir, it was making understand the concept more clearly
Can u please post some important practical questions and as well as Viva voce also
sir kindly tell how to install sqoop in windows and how to move data ms sql to hdfs using windows not other externel tools and vm etc. if already you make video related to this kindly shre link here.
Greate Sir, thanks for this .
good one sir💯