-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy path学习内容.txt
112 lines (93 loc) · 2.7 KB
/
学习内容.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
数据结构
java基础
java1.8增强：lambda+聚合操作
JAVA-Lambda
mysql
jdbc
linux
mybatis
spring
spirngmvc
ssm
maven
hadoop
hdfs
mapreduce
hive
zookeeper
hadoopha搭建
sqoop
azkaban（Oozie）
hbase
flume
nginx ---------------------D:\迅雷下载\javabigdata\光环国际大数据\13 用户行为离线项目3天\day07--用户行为分析系统离线项目\上课笔记2
git C:/Users/程都/Desktop/.git/
github chengduhaha chengdu1994
tortoisegit
egit插件
echarts
kafka
scala
IDEA IU windows+linux安装
spark core
jvm
spark sql 整合yarn 整合hive
storm
spark streaming
flink
还没学：CM软件，CDH, redis MongoDB clickhouse serverlet jsp, java成神之路，《java并发编程实战》
dbeaver（hive可视化工具）Phonenix（用Java写的基于JDBC API操作HBase的开源SQL引擎）
======================================================================
环境
<hadoop02 hadoop03 hadoop04 hadoop05>
namenode : hadoop02
yarn: hadoop04
hive: hadoop02 hadoop03 hadoop04 hadoop05
hive-->mysql: jdbc:mysql://hadoop04:3306/metastore_db_cd
zookeeper: hadoop02 hadoop03 hadoop04 hadoop05
hadoop02 hadoop03 hadoop04---->leader/follower
hadoop05---->observer
myha01
<hadoopha01 hadoopha02 hadoopha03 hadoopha04>
namenode : hadoopha01 hadoopha02
datanode hadoopha01 hadoopha02 hadoopha03 hadoopha04
yarn: hadoopha03 hadoopha04
hive: hadoopha01
hive-->mysql: jdbc:mysql://localhost:3306/myhahive01 (hadoopha01)
zookeeper: hadoopha01 hadoopha02 hadoopha03 hadoopha04
hadoopha01 hadoopha02 hadoopha03---->leader/follower
hadoopha04---->observer
journalnode hadoopha01 hadoopha02 hadoopha03
zkfc hadoopha01 hadoopha02 (与namenode相同)
sqoop hadoopha01
azkaban hadoopha01
hbase
master: hadoopha01 hadoopha02（standby）
regionserver： hadoopha01 hadoopha02 hadoopha03 hadoopha04 (与datanode相同)
flume：
高可用Flume 的 Agent 和 Collector 分布如下表所示：
名称 Host 角色
Agent1 hadoopha01 日志服务器
Agent2 hadoopha03 日志服务器
Agent3 hadoopha04 日志服务器
Collector1 hadoopha01 AgentMaster1
Collector1 hadoopha02 AgentMaster2(热备)
nginx hadoopha01
kafka hadoopha01 hadoopha02 hadoopha03 hadoopha04
scala hadoopha01 hadoopha02 hadoopha03 hadoopha04
spark master:
hadoopha01 hadoopha02
worker:
hadoopha01 hadoopha02 hadoopha03 hadoopha04
SparkHistoryServer: hadoopha01
sparksql 整合yarn 整合hive ：hadoopha01
storm nimbus：
hadoopha03 hadoopha04
supervisor：
hadoopha01 hadoopha02 hadoopha03 hadoopha04
flink
jobmanager:
hadoopha01 hadoopha04(standby)
taskmanager:
hadoopha01 hadoopha02 hadoopha03 hadoopha04
================================================================