Struct BatchLogProcessor

pub struct BatchLogProcessor { /* private fields */ }

The BatchLogProcessor collects finished logs in a buffer and exports them in batches to the configured LogExporter. This processor is ideal for high-throughput environments, as it minimizes the overhead of exporting logs individually. It uses a dedicated background thread to manage and export logs asynchronously, ensuring that the application’s main execution flow is not blocked.

This processor supports the following configurations:

• Queue size: Maximum number of log records that can be buffered.
• Batch size: Maximum number of log records to include in a single export.
• Scheduled delay: Frequency at which the batch is exported.

When using this processor with the OTLP Exporter, the following exporter features are supported:

• grpc-tonic: Requires LoggerProvider to be created within a tokio runtime.
• reqwest-blocking-client: Works with a regular main or tokio::main.

In other words, other clients like reqwest and hyper are not supported.

BatchLogProcessor buffers logs in memory and exports them in batches. An export is triggered when max_export_batch_size is reached or every scheduled_delay milliseconds. Users can explicitly trigger an export using the force_flush method. Shutdown also triggers an export of all buffered logs and is recommended to be called before the application exits to ensure all buffered logs are exported.

Warning: When using tokio’s current-thread runtime, shutdown(), which is a blocking call ,should not be called from your main thread. This can cause deadlock. Instead, call shutdown() from a separate thread or use tokio’s spawn_blocking.

Using a BatchLogProcessor:

use opentelemetry_sdk::logs::{BatchLogProcessor, BatchConfigBuilder, LoggerProvider};
use opentelemetry::global;
use std::time::Duration;
use opentelemetry_sdk::logs::InMemoryLogExporter;

let exporter = InMemoryLogExporter::default(); // Replace with an actual exporter
let processor = BatchLogProcessor::builder(exporter)
    .with_batch_config(
        BatchConfigBuilder::default()
            .with_max_queue_size(2048)
            .with_max_export_batch_size(512)
            .with_scheduled_delay(Duration::from_secs(5))
            .build(),
    )
    .build();

let provider = LoggerProvider::builder()
    .with_log_processor(processor)
    .build();

Implementations

impl BatchLogProcessor

pub fn builder<E>(exporter: E) -> BatchLogProcessorBuilder<E>

where E: LogExporter,

Create a new batch processor builder

Trait Implementations

impl Debug for BatchLogProcessor

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl LogProcessor for BatchLogProcessor

fn emit( &self, record: &mut SdkLogRecord, instrumentation: &InstrumentationScope, )

Called when a log record is ready to processed and exported.

fn force_flush(&self) -> OTelSdkResult

Force the logs lying in the cache to be exported.

fn shutdown(&self) -> OTelSdkResult

Shuts down the processor. After shutdown returns the log processor should stop processing any logs. It’s up to the implementation on when to drop the LogProcessor.

fn set_resource(&self, resource: &Resource)

Set the resource for the log processor.