I have a minor memory leak issue to solve. So that I'm wondering how can we enable profiler on KEDA. KEDA uses Operator SDK.
Go lang has a good profiler.
It is very simple to use it. Just add this line for memory profiling.
Configuration and Issue
According to the documentation, the configuration will be like this.
go get github.com/pkg/profile
then add following line to the main.go
import (
//...
"github.com/pkg/profile"
)
:
defer profile.Start(profile.MemProfile).Stop()
Run the KEDA on my local machine. However, it doesn't work. The root cause is, we need to stop the operator sdk with ctr+c. In this case, defer method doesn't execute Stop() method.
ushio@DESKTOP-KIUTRHV:~/Code/keda$ make run ARGS="--zap-log-level=debug"
/home/ushio/go/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go run \
-ldflags "-X=github.com/kedacore/keda/version.GitCommit=d0788b943ecf94f7f7c5b1a09f777cc3af6b746e -X=github.com/kedacore/keda/version.Version=v2" \
./main.go --zap-log-level=debug
2020/10/20 09:19:49 profile: memory profiling enabled (rate 4096), /tmp/profile174831365/mem.pprof
:
^C2020/10/20 09:20:02 profile: caught interrupt, stopping profiles
Exit 0 for profiling
make: *** [Makefile:96: run] Interrupt
$ ls -l /tmp/profile287657069/mem.pprof
-rw-r--r-- 1 ushio ushio 0 Oct 20 09:18 /tmp/profile287657069/mem.pprof
Solution
If you want to handle the SIG interrupt, you can use signal.Notify method for handling the signal.
func SetupCloseHandler() {
c := make(chan os.Signal)
signal.Notify(c, os.Interrupt, syscall.SIGTERM)
go func() {
<-c
fmt.Println("\r- Ctrl+C pressed in Terminal")
DeleteFiles()
os.Exit(0)
}()
}
I read the main.go they are also having the signal handling part. Let's change it to call profile.Stop() method.
func main() {
// Memory profiling
f := profile.Start(profile.MemProfile, profile.NoShutdownHook)
:
// if err := mgr.Start(ctrl.SetupSignalHandler()); err != nil {
// setupLog.Error(err, "problem running manager")
// os.Exit(1) // Profiling test
// }
fmt.Println("-----SetupCloseHandler")
setupSignalHandler := func() (stopCh <-chan struct{}) {
stop := make(chan struct{})
c := make(chan os.Signal, 2)
signal.Notify(c, os.Interrupt, syscall.SIGTERM)
go func() {
<-c
fmt.Println("Exit 0 for profiling")
f.Stop()
os.Exit(1) // second signal. Exit directly.
}()
return stop
}
if err := mgr.Start(setupSignalHandler()); err != nil {
setupLog.Error(err, "problem running manager")
os.Exit(1)
}
}
Run it. Works!
$ ls -l /tmp/profile388737932/mem.pprof
-rw-r--r-- 1 ushio ushio 47888 Oct 20 09:31 /tmp/profile388737932/mem.pprof
Viewer
The next step will be the view the analysis report.
It looks requires graphviz. Install it.
$ go tool pprof --pdf /tmp/profile356144532/mem.pprof > file.pdf
failed to execute dot. Is Graphviz installed? Error: exec: "dot": executable file not found in $PATH
$ sudo apt install graphviz
Works!
$ go tool pprof --pdf /tmp/profile356144532/mem.pprof > file.pdf
I don't know if there is better way to enable profiling using Operator SDK. Once I know it, I'll update this blog.
Top comments (0)