---
title: "Ingesting logs from Fluentd"
description: "Forward logs from Fluentd to Middleware using the Fluent Forward protocol through the Middleware Agent or an OpenTelemetry Collector."
section: "Third Party Agents"
breadcrumb: "Third Party Agents > Fluentd"
canonical: "https://docs.middleware.io/get-started/agent/fluentd"
---
# Ingesting logs from Fluentd

Middleware supports the ingestion of logs from [Fluentd](https://www.fluentd.org/). Fluentd sends logs over the [Fluent Forward protocol](https://github.com/fluent/fluentd/wiki/Forward-Protocol-Specification-v1), which is received by a Middleware Agent or an OpenTelemetry Collector and exported to your Middleware account.

```text
Fluentd ──(forward protocol)──► Middleware Agent / OTel Collector ──(OTLP)──► Middleware
```

You can use either of the following options:

- **Option 1: Middleware Agent (recommended).** The agent already listens for Fluent Forward traffic on port `8006`.
- **Option 2: OpenTelemetry Collector.** Use your own collector with the `fluentforward` receiver.

## Prerequisites

- Fluentd (or `td-agent` / `fluent-package`) installed and running.
- Your Middleware **UID** (`https://<uid>.middleware.io`) and **API key** (`MW_API_KEY`).

## Option 1: Middleware Agent (recommended)

#### 1 Install the Middleware Agent

Install the [Middleware Agent](/get-started/agent) on the host where Fluentd runs, or on a host Fluentd can reach. The agent listens for Fluent Forward traffic on port `8006` by default. To change the port, set `fluent-port` in `/etc/mw-agent/agent-config.yaml` (Linux) or the `MW_AGENT_FLUENT_PORT` environment variable, then restart the agent.

#### 2 Configure the Fluentd output

Add a `forward` output to your Fluentd configuration (usually `/etc/fluent/fluentd.conf` or `/etc/td-agent/td-agent.conf`):

```xml
<match <directive>>
  @type forward
  send_timeout 60s
  recover_wait 10s
  hard_timeout 60s

  <server>
    name middleware-agent
    host localhost
    port 8006
  </server>
</match>
```

- Replace `<directive>` with the tag pattern of the logs you want to send (for example, `app.**`, or `**` for all logs).
- If the Middleware Agent runs on a different host, replace `localhost` with that host's address.

#### 3 Restart Fluentd

```bash
sudo systemctl restart fluentd
```

Logs appear in the [Logs](/features/logs) section of your Middleware account. The `forward` output flushes its buffer every 60 seconds by default, so the first logs can take up to a minute to appear. The Fluentd tag is available as the `fluent.tag` attribute and is used as the log `source`.

## Option 2: OpenTelemetry Collector

Use this option if you already run an [OpenTelemetry Collector](/get-started/agent/otel-contrib) (contrib distribution) and do not want to install the Middleware Agent.

#### 1 Add the Fluent Forward receiver

Add the `fluentforward` receiver to your collector `config.yaml`:

```yaml
receivers:
  fluentforward:
    endpoint: 0.0.0.0:24224
```

`24224` is the default Fluent Forward port. You can change it if required. See the [receiver documentation](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/fluentforwardreceiver) for all options.

#### 2 Add the Middleware exporter and logs pipeline

```yaml
processors:
  batch:
  resourcedetection:
    detectors: [env, system]
    system:
      hostname_sources: [os]
  resource:
    attributes:
      - key: mw.account_key
        value: <MW_API_KEY>
        action: upsert
      - key: host.id
        from_attribute: host.name
        action: upsert

exporters:
  otlp/middleware:
    endpoint: https://<uid>.middleware.io:443
    headers:
      authorization: <MW_API_KEY>

service:
  pipelines:
    logs:
      receivers: [otlp, fluentforward]
      processors: [resourcedetection, resource, batch]
      exporters: [otlp/middleware]
```

Replace:

- `<uid>` with your Middleware project UID.
- `<MW_API_KEY>` with your Middleware API key.

> **Warning:** Keep the `resourcedetection` processor and the `host.id` attribute. Logs received over the Fluent Forward protocol carry no resource attributes, and Middleware rejects logs that do not identify a resource (such as `host.id` or `service.name`).

Recent collector versions log deprecation warnings for the `fluentforward` and `otlp` names. Both still work. To remove the warnings, rename them to `fluent_forward` and `otlp_grpc`.

#### 3 Configure the Fluentd output

```xml
<match <directive>>
  @type forward
  send_timeout 60s
  recover_wait 10s
  hard_timeout 60s

  <server>
    name otel-collector
    host <otel-collector-host>
    port 24224
  </server>
</match>
```

Replace `<directive>` with your tag pattern and `<otel-collector-host>` with the collector's address (`localhost` if it runs on the same host).

#### 4 Restart Fluentd and the collector

Restart both services. Logs appear in the Logs section of your Middleware account.

## Test with sample logs

To check the setup without touching your application logs, add a sample source that generates test logs:

```xml
<source>
  @type sample
  sample [{"message": "my log data", "source": "myhost"}]
  tag sample
  rate 10
</source>

<match sample>
  @type forward
  <server>
    host localhost
    port 8006
  </server>
</match>
```

Use port `24224` instead of `8006` if you followed Option 2. Remove the sample source after you see the logs in Middleware.

## Troubleshooting

- **No logs in Middleware:** check the Fluentd log (`/var/log/fluent/fluentd.log`) for connection errors to the forward target.
- **Connection refused:** make sure the agent or collector is running and that port `8006` (or `24224`) is open between Fluentd and the receiver.
- **Wrong logs are sent:** check that the `<match>` pattern matches the tags of the logs you want to forward.

Need assistance or want to learn more about Middleware? Get in touch with us via our Contact Us page or join our Slack channel.