[quagga-users 14037] Re: "SLOW THREAD" errors and "Hold Timer Expired" events on all neighbors

Andrew Gideon andrew-quagga-users-918 at tagonline.com
Wed Jul 1 20:26:45 BST 2015


On Wed, 2015-07-01 at 10:08 -0400, Andrew Gideon wrote:
> 
> I'll post the first page of the report separately.  It is taking a long
> time to load for the same memory-limiting issues that likely started
> this thread in the first place.
> 

        +   22.33%     0.00%  bgpd  [unknown]             [.] 0000000000000000
        +   16.36%    15.51%  bgpd  libzebra.so.0.0.0     [.] hash_get
        +    6.95%     5.35%  bgpd  libc-2.17.so          [.] _int_malloc
        +    5.16%     0.25%  bgpd  [kernel.kallsyms]     [k] page_fault
        +    5.09%     0.01%  bgpd  [kernel.kallsyms]     [k] system_call_fastpath
        +    4.90%     0.00%  bgpd  [kernel.kallsyms]     [k] do_page_fault
        +    4.88%     0.04%  bgpd  [kernel.kallsyms]     [k] __do_page_fault
        +    4.76%     0.13%  bgpd  [kernel.kallsyms]     [k] handle_mm_fault
        +    4.22%     2.72%  bgpd  libzebra.so.0.0.0     [.] route_lock_node
        +    3.87%     3.75%  bgpd  bgpd                  [.] 0x0000000000054dd2
        +    3.79%     0.29%  bgpd  libc-2.17.so          [.] __vsnprintf_chk
        +    3.64%     3.50%  bgpd  bgpd                  [.] 0x0000000000033b72
        +    3.35%     3.18%  bgpd  libzebra.so.0.0.0     [.] route_node_get
        +    3.20%     0.01%  bgpd  [kernel.kallsyms]     [k] swapin_readahead
        +    2.82%     0.03%  bgpd  libc-2.17.so          [.] __select
        +    2.66%     0.03%  bgpd  [kernel.kallsyms]     [k] sys_select
        +    2.50%     0.06%  bgpd  [kernel.kallsyms]     [k] core_sys_select
        +    2.37%     0.21%  bgpd  [kernel.kallsyms]     [k] do_select
        +    2.34%     2.27%  bgpd  bgpd                  [.] 0x0000000000033b34
        +    2.24%     0.10%  bgpd  [kernel.kallsyms]     [k] apic_timer_interrupt
        +    2.20%     2.15%  bgpd  libzebra.so.0.0.0     [.] prefix_match
        +    2.18%     2.01%  bgpd  libc-2.17.so          [.] malloc_consolidate
        +    2.13%     0.02%  bgpd  [kernel.kallsyms]     [k] smp_apic_timer_interrupt
        +    2.12%     2.05%  bgpd  libc-2.17.so          [.] __libc_calloc
        +    1.99%     1.92%  bgpd  libc-2.17.so          [.] _int_free
        +    1.84%     0.02%  bgpd  [kernel.kallsyms]     [k] local_apic_timer_interrupt
        +    1.79%     0.06%  bgpd  [kernel.kallsyms]     [k] hrtimer_interrupt
        +    1.79%     0.00%  bgpd  [kernel.kallsyms]     [k] blk_finish_plug
        +    1.79%     0.01%  bgpd  [kernel.kallsyms]     [k] blk_flush_plug_list
        +    1.76%     1.68%  bgpd  libc-2.17.so          [.] vfprintf
        +    1.72%     0.00%  bgpd  [kernel.kallsyms]     [k] __blk_run_queue
        +    1.71%     0.01%  bgpd  [kernel.kallsyms]     [k] scsi_request_fn
        +    1.66%     0.02%  bgpd  [kernel.kallsyms]     [k] __run_hrtimer
        +    1.64%     1.55%  bgpd  libzebra.so.0.0.0     [.] hash_release
        +    1.54%     0.00%  bgpd  [kernel.kallsyms]     [k] scsi_dispatch_cmd
        +    1.53%     0.01%  bgpd  [kernel.kallsyms]     [k] ata_scsi_queuecmd
        +    1.50%     0.01%  bgpd  [kernel.kallsyms]     [k] tick_sched_timer

Expanding the top two:

        -   22.33%     0.00%  bgpd  [unknown]             [.] 0000000000000000                                                     ◆
           - 0                                                                                                                     ▒
              - 3.90% 0x7f526de31ad0                                                                                               ▒
                   68.07% 0x7f526d914530                                                                                           ▒
                   31.93% 0x7f526d929a60                                                                                           ▒
              - 2.30% 0x7f526d81ac20                                                                                               ▒
                   0x7f526d914530                                                                                                  ▒
              - 2.03% 0x7f526d9da970                                                                                               ▒
                   74.25% 0x7f526d929a60                                                                                           ▒
                   25.75% 0x7f526d914530                                                                                           ▒
              - 1.86% 0x7f526d9c7100                                                                                               ▒
                   0x7f526d914530                                                                                                  ▒
              - 1.79% 0x7f526d93eab0                                                                                               ▒
                   0x7f526d914530                                                                                                  ▒
              - 1.78% 0x7f526d979190                                                                                               ▒
                   93.41% 0x7f526d929a60                                                                                           ▒
                   6.59% 0x7f526d914530                                                                                            ▒
              - 1.75% 0x7f526da28b30                                                                                               ▒
                   90.51% 0x7f526d929a60                                                                                           ▒
                   9.49% 0x7f526d914530                                                                                            ▒
              - 1.55% 0x7f5276168ce0                                                                                               ▒
                   0x7f526d914530                                                                                                  ▒
              - 1.51% 0x7f526d9659f0                                                                                               ▒
                   0x7f526d914530                                                                                                  ▒
              - 1.50% 0x7f526da152c0                                                                                               ▒
                   0x7f526d914530                                                                                                  ▒
              - 1.03% 0x7f52786df910                                                                                               ▒
                   0x7f526d914530                                                                                                  ▒
              - 0.88% 0x7f526da45800                                                                                               ▒
                   0x7f526d914530                                                                                                  ▒
              - 0.63% 0x7f5279de9e60                                                                                               ▒
                   0x7f526d914530                                                                                                  ▒
              - 0.63% 0x7f526da153b0                                                                                               ▒
                   0x7f526d914530                                                                                                  ▒
        
        
        
        
        
        -   16.36%    15.51%  bgpd  libzebra.so.0.0.0     [.] hash_get                                                             ▒
           - hash_get                                                                                                              ▒
              - 38.07% 0                                                                                                           ▒
                   20.68% 0x8b4818ec83485355                                                                                       ▒
                   11.47% 0xf628478b48fb8948                                                                                       ▒
                 - 11.44% 0x7f52881fab50                                                                                           ▒
                      0x7f527f104f90                                                                                               ▒
                 - 11.42% 0x7f52846fd200                                                                                           ▒
                      0x7f5280eea660                                                                                               ▒
                   11.41% 0x88ec81485355                                                                                           ▒
                 - 11.40% 0x7f5284eb0420                                                                                           ▒
                      0x7f52807c5d30                                                                                               ▒
                 - 11.17% 0x7f52870a1e00                                                                                           ▒
                      0x7f5281e08fc0                                                                                               ▒
                   11.01% 0xfffffff0358d4800                                                                                       ▒
                4.55% 0x30                                                                                                         ▒
                2.92% 0x2c                                                                                                         ▒
                2.60% 0x4                                                                                                          ▒
                2.56% 0x34                                                                                                         ▒
                0.65% 0x48                                                                                                         ▒
                0.65% 0x38                                      
        
        
	- Andrew





More information about the Quagga-users mailing list