netstat를 이용하여 전체 Network 의 상태를 파악이 가능하지만, 세부적인 통신내용을 알기는 어려운 것 같다.
$ netstat --help usage: netstat [-vWeenNcCF] [] -r netstat {-V|--version|-h|--help} netstat [-vWnNcaeol] [ ...] netstat { [-vWeenNac] -i | [-cWnNe] -M | -s } -r, --route display routing table -i, --interfaces display interface table -g, --groups display multicast group memberships -s, --statistics display networking statistics (like SNMP) -M, --masquerade display masqueraded connections -v, --verbose be verbose -W, --wide don't truncate IP addresses -n, --numeric don't resolve names --numeric-hosts don't resolve host names --numeric-ports don't resolve port names --numeric-users don't resolve user names -N, --symbolic resolve hardware names -e, --extend display other/more information -p, --programs display PID/Program name for sockets -c, --continuous continuous listing -l, --listening display listening server sockets -a, --all, --listening display all sockets (default: connected) -o, --timers display timers -F, --fib display Forwarding Information Base (default) -C, --cache display routing cache instead of FIB ={-t|--tcp} {-u|--udp} {-w|--raw} {-x|--unix} --ax25 --ipx --netrom =Use '-6|-4' or '-A ' or '-- '; default: inet List of possible address families (which support routing): inet (DARPA Internet) inet6 (IPv6) ax25 (AMPR AX.25) netrom (AMPR NET/ROM) ipx (Novell IPX) ddp (Appletalk DDP) x25 (CCITT X.25)
1.1 netstat 기본사용법
사용방법은 Network의 전체 구성인 Routing Table 정보부터 보는 것이 맞을 것 같으며, 관련된 Multicast 와 점점 세부 네트워크로 들어가 분석하는 것이 맞을 것 같다.
- Routing Table 정보 확인
$ netstat -re // extened or $ netstat -r Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface default router.asus.com 0.0.0.0 UG 0 0 0 eth0 10.0.0.0 * 255.255.255.0 U 0 0 0 eth0 link-local * 255.255.0.0 U 0 0 0 usb0 172.17.0.0 * 255.255.0.0 U 0 0 0 docker0 192.168.55.0 * 255.255.255.0 U 0 0 0 l4tbr0 192.168.55.0 * 255.255.255.0 U 0 0 0 usb0
- Multicast 관련 정보 확인
$ netstat -g IPv6/IPv4 Group Memberships Interface RefCnt Group --------------- ------ --------------------- lo 1 all-systems.mcast.net eth0 1 224.0.0.251 eth0 1 all-systems.mcast.net usb0 1 224.0.0.251 usb0 1 all-systems.mcast.net usb1 1 all-systems.mcast.net l4tbr0 1 224.0.0.251 l4tbr0 1 all-systems.mcast.net wlan0 1 all-systems.mcast.net docker0 1 224.0.0.251 docker0 1 all-systems.mcast.net lo 1 ff02::1 lo 1 ff01::1 dummy0 1 ff02::1 dummy0 1 ff01::1 eth0 1 ff02::fb eth0 1 ff02::1:fff4:9ad0 eth0 1 ff02::1 eth0 1 ff01::1 tunl0 1 ff02::1 tunl0 1 ff01::1 usb0 1 ff02::fb usb0 1 ff02::1:ffe6:405f usb0 1 ff02::1 usb0 1 ff01::1 usb1 1 ff02::fb usb1 1 ff02::1:ff72:214 usb1 1 ff02::1 usb1 1 ff01::1 l4tbr0 1 ff02::fb l4tbr0 1 ff02::1:ff2a:6c1f l4tbr0 1 ff02::1 l4tbr0 1 ff01::1 wlan0 1 ff02::1 wlan0 1 ff01::1 docker0 1 ff02::1 docker0 1 ff01::1
- TCP/UDP 관련된 모든 Socket 상태정보 표시
옵션에따라 점점 추가 되며 아래와 같이 파악 가능하다.
$ netstat -atn # all socket and numeric For tcp $ netstat -aun # all socket and numeric For udp $ netstat -atun # all socket and numeric For both (TCP,UDP) // 각 State 와 PID 정보 및 Protocol 과 Recv Q/Send Q 정보 $ netstat -tulnp # listening server and numeric and PID for both (TCP,UDP) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 127.0.1.1:53 0.0.0.0:* LISTEN - tcp 0 0 192.168.55.2:53 0.0.0.0:* LISTEN - tcp 0 0 127.0.0.1:53 0.0.0.0:* LISTEN - tcp 0 0 192.168.55.1:53 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN - tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN - tcp 0 0 127.0.0.1:6011 0.0.0.0:* LISTEN - tcp 0 0 127.0.0.1:6012 0.0.0.0:* LISTEN - tcp 0 0 127.0.0.1:2947 0.0.0.0:* LISTEN - tcp6 0 0 fe80::c03:7dff:fea1::53 :::* LISTEN - tcp6 0 0 ::1:53 :::* LISTEN - tcp6 0 0 :::22 :::* LISTEN - tcp6 0 0 ::1:631 :::* LISTEN - tcp6 0 0 ::1:6011 :::* LISTEN - tcp6 0 0 ::1:6012 :::* LISTEN - tcp6 0 0 ::1:2947 :::* LISTEN - udp 0 0 0.0.0.0:5353 0.0.0.0:* - udp 12585600 0 0.0.0.0:1334 0.0.0.0:* 1695/python udp 0 0 0.0.0.0:52316 0.0.0.0:* - udp 0 0 127.0.1.1:53 0.0.0.0:* - udp 0 0 192.168.55.2:53 0.0.0.0:* - udp 0 0 127.0.0.1:53 0.0.0.0:* - udp 0 0 192.168.55.1:53 0.0.0.0:* - udp 0 0 0.0.0.0:68 0.0.0.0:* - udp 0 0 0.0.0.0:631 0.0.0.0:* - udp 0 0 0.0.0.0:53992 0.0.0.0:* - udp6 0 0 :::5353 :::* - udp6 0 0 :::51705 :::* - udp6 0 0 fe80::c03:7dff:fea1::53 :::* - udp6 0 0 ::1:53 :::* -
https://askubuntu.com/questions/538208/how-to-check-opened-closed-ports-on-my-computer
- network Interface의 정보확인 ( Layer 2 분석)
Network TEST를 통해서 기본확인가능하며, Kernel의 Driver에 따라 에러감지가 제대로 되지 않을 수 있으므로 주의하자
만약 에러가 발생하면, ifconfig에서도 쉽게 발견가능
netstat -i, --interfaces Display a table of all network interfaces.
$ netstat -i or netstat -ic // Interface TX/RX Status Kernel Interface table Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg docker0 1500 0 0 0 0 0 0 0 0 0 BMU eth0 1500 0 2529153 0 0 0 829553 0 0 0 BMRU l4tbr0 1500 0 127916 0 0 0 103336 0 0 0 BMRU lo 65536 0 3462 0 0 0 3462 0 0 0 LRU usb0 1500 0 33527 33527 0 0 32753 0 0 0 BMRU usb1 1500 0 94389 0 0 0 70958 0 0 0 BMRU wlan0 1500 0 0 0 0 0 0 0 0 0 BMU
- MTU(Maximum Transfer Unit): default 1500 // ethernet
- Met: 메트릭
- RX-OK : 받은 것. 정상 수신 패킷
- RX-ERR: 파손 수신 패킷 (상위 usb0은 100% 에러)
- RX-DRP: 유실 수신 패킷
- RX-OVR: packet overrun(커널에서 처리 하는 인터럽트보다 패킷이 빨리 도착했을때 패킷 유실)
- TX-OK : 보낸 것. 정상 송신 패킷.
- TX-ERR: 파손 송신 패킷.
- TX-DRP: 유실 송신 패킷.
- TX-OVR: packet overrun
- Flg:
- B: 브로드케스트 주소 지정
- L: 루프백 인터페이스
- M: 모든 패킷 수집(promiscuous mode)
- O: 인터페이스에 대한 ARP가 off 되어 있음.
- P: point-to-point 연결
- R: 인터페이스 동작중(Running)
- U: 인터페이스 up 인 상태.
2. Network 문제사항 및 에러확인
Network 관련설정은 처음 booting /etc/sysctrl.conf에 의해서 설정되며 없다면, kernel의 default 값으로 설정된다.
Network 정보 (TCP / UDP / IP / ICMP) 의 관련된사항들을 기록하였기때문에, 분석이 가능하며, 문제사항을 쉽게 파악이 가능하다.
netstat -s 옵션이외에도 proc 정보로도 확인가능
sysctl -a 설정정보확인가능
netstat -s , --statistics Display summary statistics for each protocol.
$ netstat -s // UDP / TCP / ICMP 사용통계 및 문제에러사항 확인
Ip:
1807814 total packets received
0 forwarded
0 incoming packets discarded
1188470 incoming packets delivered
955549 requests sent out
40 outgoing packets dropped
Icmp:
100 ICMP messages received
2 input ICMP message failed.
ICMP input histogram:
destination unreachable: 100
102 ICMP messages sent
0 ICMP messages failed
ICMP output histogram:
destination unreachable: 102
IcmpMsg:
InType3: 100
OutType3: 102
Tcp:
5894 active connections openings
15 passive connection openings
4 failed connection attempts
5748 connection resets received
6 connections established
965515 segments received
952551 segments send out
146 segments retransmited
0 bad segments received.
34 resets sent
Udp:
59434 packets received
82 packets to unknown port received.
0 packet receive errors
1454 packets sent
IgnoredMulti: 139426
UdpLite:
TcpExt:
102 TCP sockets finished time wait in fast timer
12122 delayed acks sent
38 delayed acks further delayed because of locked socket
Quick ack mode was activated 8 times
404422 packets directly queued to recvmsg prequeue.
36435935 bytes directly in process context from backlog
434921871 bytes directly received in process context from prequeue
152208 packet headers predicted
344744 packets header predicted and directly queued to user
1419 acknowledgments not containing data payload received
284057 predicted acknowledgments
41 congestion windows recovered without slow start after partial ack
36 timeouts in loss state
1 retransmits in slow start
46 other TCP timeouts
TCPLossProbes: 2
4 DSACKs sent for old packets
12 connections reset due to unexpected data
3 connections reset due to early user close
4 connections aborted due to timeout
IPReversePathFilter: 194
TCPRcvCoalesce: 104075
TCPOFOQueue: 2614
TCPOFOMerge: 3
TCPChallengeACK: 27
TCPSpuriousRtxHostQueues: 2
TCPAutoCorking: 68999
TCPFromZeroWindowAdv: 15
TCPToZeroWindowAdv: 22
TCPWantZeroWindowAdv: 122
TCPSynRetrans: 4
TCPOrigDataSent: 561796
TCPHystartTrainDetect: 3
TCPHystartTrainCwnd: 50
TCPACKSkippedChallenge: 31
TCPKeepAlive: 2
IpExt:
InNoRoutes: 1
InMcastPkts: 82733
OutMcastPkts: 2070
InBcastPkts: 139441
OutBcastPkts: 18
InOctets: 976384465
OutOctets: 295103139
InMcastOctets: 8631056
OutMcastOctets: 99130
InBcastOctets: 17773399
OutBcastOctets: 812
InNoECTPkts: 1819528
- UDP 만 별도 문제사항확인
혹은 net.core.rmem_max 와 net.core.rmem_default 변경을 해주는 것인데, 이것보다는 위의 것이 더 나은 방법인 것 같다.
(/etc/sysctrl.conf 에서 변경가능)
$ netstat -su //UDP
IcmpMsg:
InType3: 84
OutType3: 1411
Udp:
119049 packets received
2339998 packets to unknown port received.
6006884 packet receive errors
2618 packets sent
RcvbufErrors: 6006884 //Recv Buffer Error , cat /proc/net/snmp | grep Udp 값 동일
IgnoredMulti: 494148
UdpLite:
IpExt:
InNoRoutes: 1
InMcastPkts: 170389
OutMcastPkts: 6945
InBcastPkts: 425852
OutBcastPkts: 18
InOctets: 14776499147
OutOctets: 3391504334
InMcastOctets: 11684052
OutMcastOctets: 279654
InBcastOctets: 39906199
OutBcastOctets: 812
InNoECTPkts: 12363330
UDP Packet Drop 현상 ( Test1, Test2 의 에러율 확인)
https://stackoverflow.com/questions/21664950/udp-packet-drop-inerrors-vs-rcvbuferrors
- Socket Buffer 설정정보확인 (proc)
$ ls /proc/sys/net/core/ busy_poll dev_weight max_skb_frags netdev_budget netdev_tstamp_prequeue rmem_max tstamp_allow_data wmem_max xfrm_aevent_rseqth busy_read flow_limit_cpu_bitmap message_burst netdev_max_backlog optmem_max rps_sock_flow_entries warnings xfrm_acq_expires xfrm_larval_drop default_qdisc flow_limit_table_len message_cost netdev_rss_key rmem_default somaxconn wmem_default xfrm_aevent_etime //Socket Recv Buffer (Memory) $ cat /proc/sys/net/core/rmem_default //Socket Recv Buffer Default 229376 $ cat /proc/sys/net/core/rmem_max // Socket Recv Buffer Max 16777216 //Socket Write Buffer (Memory) $ cat /proc/sys/net/core/wmem_default //Socket Write Buffer Default $ cat /proc/sys/net/core/wmem_max // Socket Write Buffer Max
- IP기반의 protocol 정보
$ls /proc/sys/net/ipv4/ conf/ ipfrag_high_thresh tcp_fin_timeout tcp_reordering fwmark_reflect ipfrag_low_thresh tcp_frto tcp_retrans_collapse icmp_echo_ignore_all ipfrag_max_dist tcp_fwmark_accept tcp_retries1 icmp_echo_ignore_broadcasts ipfrag_secret_interval tcp_invalid_ratelimit tcp_retries2 icmp_errors_use_inbound_ifaddr ipfrag_time tcp_keepalive_intvl tcp_rfc1337 icmp_ignore_bogus_error_responses neigh/ tcp_keepalive_probes tcp_rmem icmp_msgs_burst netfilter/ tcp_keepalive_time tcp_sack icmp_msgs_per_sec ping_group_range tcp_limit_output_bytes tcp_slow_start_after_idle icmp_ratelimit route/ tcp_low_latency tcp_stdurg icmp_ratemask tcp_abort_on_overflow tcp_max_orphans tcp_syn_retries igmp_link_local_mcast_reports tcp_adv_win_scale tcp_max_reordering tcp_synack_retries igmp_max_memberships tcp_allowed_congestion_control tcp_max_syn_backlog tcp_thin_dupack igmp_max_msf tcp_app_win tcp_max_tw_buckets tcp_thin_linear_timeouts igmp_qrv tcp_autocorking tcp_mem tcp_timestamps inet_peer_maxttl tcp_available_congestion_control tcp_min_rtt_wlen tcp_tso_win_divisor inet_peer_minttl tcp_base_mss tcp_min_tso_segs tcp_tw_recycle inet_peer_threshold tcp_challenge_ack_limit tcp_moderate_rcvbuf tcp_tw_reuse ip_default_ttl tcp_congestion_control tcp_mtu_probing tcp_window_scaling ip_dynaddr tcp_default_init_rwnd tcp_no_metrics_save tcp_wmem ip_early_demux tcp_dsack tcp_notsent_lowat tcp_workaround_signed_windows ip_forward tcp_early_retrans tcp_orphan_retries udp_mem ip_forward_use_pmtu tcp_ecn tcp_pacing_ca_ratio udp_rmem_min ip_local_port_range tcp_ecn_fallback tcp_pacing_ss_ratio udp_wmem_min ip_local_reserved_ports tcp_fack tcp_probe_interval vs/ ip_no_pmtu_disc tcp_fastopen tcp_probe_threshold xfrm4_gc_thresh $ cat /proc/sys/net/ipv4/tcp_mem 94236 125648 188472
- SNMP 관련정보
- TCP 관련부분 분석
$ cat /proc/net/snmp Ip: Forwarding DefaultTTL InReceives InHdrErrors InAddrErrors ForwDatagrams InUnknownProtos InDiscards InDelivers OutRequests OutDiscards OutNoRoutes ReasmTimeout ReasmReqds ReasmOKs ReasmFails FragOKs FragFails FragCreates Ip: 1 64 12843660 0 0 0 0 0 12843660 22324680 0 0 0 0 0 0 0 0 0 Icmp: InMsgs InErrors InCsumErrors InDestUnreachs InTimeExcds InParmProbs InSrcQuenchs InRedirects InEchos InEchoReps InTimestamps InTimestampReps InAddrMasks InAddrMaskReps OutMsgs OutErrors OutDestUnreachs OutTimeExcds OutParmProbs OutSrcQuenchs OutRedirects OutEchos OutEchoReps OutTimestamps OutTimestampReps OutAddrMasks OutAddrMaskReps Icmp: 1 0 0 1 0 0 0 0 0 0 0 0 0 0 4 0 4 0 0 0 0 0 0 0 0 0 0 IcmpMsg: InType3 OutType3 IcmpMsg: 1 4 Tcp: RtoAlgorithm RtoMin RtoMax MaxConn ActiveOpens PassiveOpens AttemptFails EstabResets CurrEstab InSegs OutSegs RetransSegs InErrs OutRsts InCsumErrors Tcp: 1 200 120000 -1 224248 11 2 3 3 12799177 22311554 11 0 15 0 Udp: InDatagrams NoPorts InErrors OutDatagrams RcvbufErrors SndbufErrors InCsumErrors IgnoredMulti Udp: 8452 3 0 3600 0 0 0 70 UdpLite: InDatagrams NoPorts InErrors OutDatagrams RcvbufErrors SndbufErrors InCsumErrors IgnoredMulti UdpLite: 0 0 0 0 0 0 0 0
- UDP RcvBuffErrors 분석
RecvbufErrors가 지속적으로 갑자기 증가되는 문제발생하여 관련된 부분 디버깅
$ netstat -su
IcmpMsg:
InType3: 88
OutType3: 88
Udp:
6404859 packets received
88 packets to unknown port received.
2295229 packet receive errors
1148 packets sent
RcvbufErrors: 2295229
IgnoredMulti: 281547
UdpLite:
IpExt:
InMcastPkts: 10936
OutMcastPkts: 2477
InBcastPkts: 275312
OutBcastPkts: 18
InOctets: 12773054136
OutOctets: 240440435
InMcastOctets: 379933
OutMcastOctets: 102955
InBcastOctets: 23165236
OutBcastOctets: 812
InNoECTPkts: 9232445
$ cat /proc/net/snmp
Ip: Forwarding DefaultTTL InReceives InHdrErrors InAddrErrors ForwDatagrams InUnknownProtos InDiscards InDelivers OutRequests OutDiscards OutNoRoutes ReasmTimeout ReasmReqds ReasmOKs ReasmFails FragOKs FragFails FragCreates
Ip: 1 64 7026190 0 1 0 0 0 7025803 300352 40 0 0 0 0 0 0 0 0
Icmp: InMsgs InErrors InCsumErrors InDestUnreachs InTimeExcds InParmProbs InSrcQuenchs InRedirects InEchos InEchoReps InTimestamps InTimestampReps InAddrMasks InAddrMaskReps OutMsgs OutErrors OutDestUnreachs OutTimeExcds OutParmProbs OutSrcQuenchs OutRedirects OutEchos OutEchoReps OutTimestamps OutTimestampReps OutAddrMasks OutAddrMaskReps
Icmp: 88 0 0 88 0 0 0 0 0 0 0 0 0 0 88 0 88 0 0 0 0 0 0 0 0 0 0
IcmpMsg: InType3 OutType3
IcmpMsg: 88 88
Tcp: RtoAlgorithm RtoMin RtoMax MaxConn ActiveOpens PassiveOpens AttemptFails EstabResets CurrEstab InSegs OutSegs RetransSegs InErrs OutRsts InCsumErrors
Tcp: 1 200 120000 -1 37 5 2 6 3 190165 374801 39 0 25 0
Udp: InDatagrams NoPorts InErrors OutDatagrams RcvbufErrors SndbufErrors InCsumErrors IgnoredMulti
Udp: 6155446 88 466455 1007 466455 0 0 206305
UdpLite: InDatagrams NoPorts InErrors OutDatagrams RcvbufErrors SndbufErrors InCsumErrors IgnoredMulti
UdpLite: 0 0 0 0 0 0 0 0
$ cat /proc/1695/net/udp // RX QUEUE를 Socket으로 6x1024x1024로 설정했지만, 아래와 같이 12x1024x1024로 증가되었음
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode ref pointer drops
229: 00000000:14E9 00000000:0000 07 00000000:00000000 00:00000000 00000000 110 0 33308 2 0000000000000000 0
306: 00000000:0536 00000000:0000 07 00000000:00C00300 00:00000000 00000000 1001 0 37343 2 0000000000000000 2295417 //RcvbufErrors 동일값
2136: 00000000:CC5C 00000000:0000 07 00000000:00000000 00:00000000 00000000 65534 0 34055 2 0000000000000000 0
3121: 0101007F:0035 00000000:0000 07 00000000:00000000 00:00000000 00000000 0 0 34043 2 0000000000000000 0
3121: 0237A8C0:0035 00000000:0000 07 00000000:00000000 00:00000000 00000000 111 0 24293 2 0000000000000000 0
3121: 0100007F:0035 00000000:0000 07 00000000:00000000 00:00000000 00000000 0 0 29242 2 0000000000000000 0
3121: 0137A8C0:0035 00000000:0000 07 00000000:00000000 00:00000000 00000000 0 0 29240 2 0000000000000000 0
3136: 00000000:0044 00000000:0000 07 00000000:00000000 00:00000000 00000000 0 0 300750 2 0000000000000000 0
3699: 00000000:0277 00000000:0000 07 00000000:00000000 00:00000000 00000000 0 0 321449 2 0000000000000000 0
3812: 00000000:D2E8 00000000:0000 07 00000000:00000000 00:00000000 00000000 110 0 33310 2 0000000000000000 0
$ cat /proc/1695/oom_score
193
$ netstat -aun
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
udp 0 0 0.0.0.0:5353 0.0.0.0:*
udp 12585728 0 0.0.0.0:1334 0.0.0.0:* //00C00300 : 12583680 과 동일한 수치지만, 지속적으로 증가
udp 0 0 0.0.0.0:52316 0.0.0.0:*
udp 0 0 127.0.1.1:53 0.0.0.0:*
udp 0 0 192.168.55.2:53 0.0.0.0:*
udp 0 0 127.0.0.1:53 0.0.0.0:*
udp 0 0 192.168.55.1:53 0.0.0.0:*
udp 0 0 0.0.0.0:68 0.0.0.0:*
udp 0 0 0.0.0.0:631 0.0.0.0:*
udp 0 0 0.0.0.0:53992 0.0.0.0:*
udp6 0 0 :::5353 :::*
udp6 0 0 :::51705 :::*
udp6 0 0 fe80::c03:7dff:fea1::53 :::*
udp6 0 0 ::1:53 :::*
SOCKET의 SO_RCVBUF 설정
https://serverfault.com/questions/758885/rx-queue-value-in-proc-net-udp-grows-to-twice-the-so-rcvbuf-value
2.1 OS 및 Socket 관련 세부설정
상위 proc에서 확인된 socket 내부 buffer 설정을 linux에서는 sysctl 명령어로 변경이 가능하며, 이외 buffer 설정이외에도, 다양한 설정이 가능하다. 이부분은 별도의 Manual을 참조하자.
- Socket 관련된 memory 설정값 확인
$ sysctl -a | grep mem sysctl: permission denied on key 'fs.protected_hardlinks' sysctl: permission denied on key 'fs.protected_symlinks' sysctl: permission denied on key 'kernel.cad_pid' sysctl: permission denied on key 'kernel.usermodehelper.bset' ..... //229376 = 224 * 1024 //6291456 = 6 * 1024 * 1024 //3145728 = 3 * 1024 * 1024 //16777216 = 16 * 1024 *1024 //12583680 = 12 * 1024 * 1024 //26214400 = 25 * 1024*1024 net.core.optmem_max = 20480 net.core.rmem_default = 229376 // Byte (UDP/TCP) net.core.rmem_max = 229376 // Byte (UDP/TCP) net.core.wmem_default = 229376 // Byte (UDP/TCP) net.core.wmem_max = 229376 // Byte (UDP/TCP) net.ipv4.igmp_max_memberships = 20 sysctl: permission denied on key 'net.ipv4.tcp_fastopen_key' sysctl: permission denied on key 'net.ipv6.conf.all.stable_secret' net.ipv4.tcp_mem = 94236 125648 188472 // Page(4KB) , min / pressure / max net.ipv4.tcp_rmem = 4096 87380 6291456 // Byte , min / pressure / max net.ipv4.tcp_wmem = 4096 16384 4194304 // Byte , min / pressure / max net.ipv4.udp_mem = 188472 251297 376944 // Page(4KB) min / pressure / max net.ipv4.udp_rmem_min = 4096 net.ipv4.udp_wmem_min = 4096 net.ipv4.vs.amemthresh = 1024 sysctl: permission denied on key 'net.ipv6.conf.default.stable_secret' sysctl: permission denied on key 'net.ipv6.conf.docker0.stable_secret' sysctl: permission denied on key 'net.ipv6.conf.dummy0.stable_secret' sysctl: permission denied on key 'net.ipv6.conf.eth0.stable_secret' sysctl: permission denied on key 'net.ipv6.conf.l4tbr0.stable_secret' sysctl: permission denied on key 'net.ipv6.conf.lo.stable_secret' sysctl: permission denied on key 'net.ipv6.conf.tunl0.stable_secret' sysctl: permission denied on key 'net.ipv6.conf.usb0.stable_secret' sysctl: permission denied on key 'net.ipv6.conf.usb1.stable_secret' sysctl: permission denied on key 'net.ipv6.conf.wlan0.stable_secret' sysctl: permission denied on key 'vm.mmap_rnd_bits' sysctl: permission denied on key 'vm.mmap_rnd_compat_bits' vm.lowmem_reserve_ratio = 256 32 vm.overcommit_memory = 0
- Kernel의 내부 Socket Buffer 변경
$ sysctl -w net.core.rmem_default="229376" $ sysctl -w net.core.wmem_default="229376" $ sysctl -w net.core.rmem_max="16777216" $ sysctl -w net.core.wmem_max="16777216" $ sysctl -w net.ipv4.tcp_rmem="229376 229376 16777216" $ sysctl -w net.ipv4.tcp_wmem="229376 229376 16777216"
Recvbuffer 만 변경하여 테스트 진행
$ sudo sysctl -w net.core.rmem_default=26214400 $ sudo sysctl -w net.core.rmem_max=26214400
- /etc/sysctl.conf 에서 기본설정
$ sudo vi /etc/sysctl.conf ........ net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.ipv4.tcp_rmem = 4096 229376 16777216 net.ipv4.tcp_wmem = 4096 229376 16777216 ........ $ ls /etc/sysctl.d/ // 확인man sysctl.conf 참조
/proc 및 /sys 정보관련정보
https://www.kernel.org/doc/Documentation/sysctl/net.txt
http://www.ubuntugeek.com/performance-tuning-with-system-control-sysctl-in-ubuntu.html
https://kensei.co.kr/289
https://rocksea.tistory.com/64
https://meetup.toast.com/posts/53
https://wwwx.cs.unc.edu/~sparkst/howto/network_tuning.php
https://www.systutorials.com/241303/how-to-enlarge-linux-udp-buffer-size/
https://m.blog.naver.com/PostView.nhn?blogId=parkjy76&logNo=220874390930&proxyReferer=https%3A%2F%2Fwww.google.com%2F
https://stackoverflow.com/questions/31546835/tcp-receiving-window-size-higher-than-net-core-rmem-max
https://m.blog.naver.com/PostView.nhn?blogId=kwoncharlie&logNo=10109682872&proxyReferer=https%3A%2F%2Fwww.google.com%2F
- watch or shell script 를 이용하여 감시
$ sudo watch -d -n0 "netstat -atnp | grep ESTA" // -d, difference , -n interval second , grep ESTABLISHED 된것을 확인 $ man watch // 세부명령어 확인
TCP 의 Socket 상태도
https://upload.wikimedia.org/wikipedia/commons/e/e4/Tcp_state_diagram.svg
