kafka 서버에 SSL 혹은 SASL 을 적용하면 성능 저하가 있을까?

datasheet.xlsx

0.01MB

SSL

SSL의 경우 이미 리포트가 있어서 결과만 공유한다.

https://blog.mimacom.com/apache-kafka-with-ssltls-performance/

Performance considerations when using Apache Kafka with SSL/TLS

 

테스트 환경 SPEC
Number of cluster nodes: 3
Environment: Amazon EC2
Instance type: m3.medium
Operating system: Ubuntu Server 16.04 LTS
Number of CPUs: 1
Memory: 3,75GB
JDK: Oracle  8u111
Topic replication factor: 3
Topic partitions: 3

 

결론 : SSL handshake 를 수행하게 되면 23% 성능저하를 유발(당연한 결과)

1. Running a performance test with this cluster without SSL yields the following results:

Records sent: 10000000
Records/s: 174285
Throughput: 166,21 MB/sec
Latency avg: 6271,86 ms
Latency max: 9065,00 ms

 

 

2. In comparison active SSL/TLS gives the following results:

Records sent: 10000000
Records/s: 133459
Throughput: 127,28 MB/sec
Latency avg: 5911,68 ms
Latency max: 12526,00 ms

---

SASL(kerberos, keytab)

서버 SPEC : 4CPU, 8GB MEM, 3 Replica

 

1. producer 성능비교

결론 : SASL 방식이 keytab 파일을 참조해서 Disk IO가 좀더 발생하고 트랜젝션 지연 거의 없음

 

가. NORMAL 통신

52 records sent, 10.3 records/sec (0.01 MB/sec), 20.3 ms avg latency, 538.0 ms max latency.

50 records sent, 9.9 records/sec (0.01 MB/sec), 6.1 ms avg latency, 9.0 ms max latency.

50 records sent, 9.9 records/sec (0.01 MB/sec), 5.5 ms avg latency, 9.0 ms max latency.

51 records sent, 10.1 records/sec (0.01 MB/sec), 4.9 ms avg latency, 7.0 ms max latency.

 

나. SASL 통신

51 records sent, 10.2 records/sec (0.01 MB/sec), 22.8 ms avg latency, 613.0 ms max latency.

51 records sent, 10.1 records/sec (0.01 MB/sec), 5.0 ms avg latency, 8.0 ms max latency.

50 records sent, 9.9 records/sec (0.01 MB/sec), 4.6 ms avg latency, 6.0 ms max latency.

50 records sent, 10.0 records/sec (0.01 MB/sec), 4.5 ms avg latency, 7.0 ms max latency.

 

 

2. consumer 성능비교

결론 : 컨슈밍의 경우 성능저하가 있으며 약 7%

	data.consumed.in.MB, MB.sec, data.consumed.in.nMsg, nMsg.sec, rebalance.time.ms, fetch.time.ms, fetch.MB.sec, fetch.nMsg.sec
NORMAL 통신	0.4806, 0.6307, 506, 664.0420, 583, 179, 2.6849, 2826.8156
SASL 통신	0.4806, 0.5847, 506, 615.5718, 645, 177, 2.7153, 2858.7571