Skip to content

Feature: Enhanced Media Sending#9

Open
moothz wants to merge 2 commits intoEvolutionAPI:mainfrom
moothz:feature/enhanced-media-sending
Open

Feature: Enhanced Media Sending#9
moothz wants to merge 2 commits intoEvolutionAPI:mainfrom
moothz:feature/enhanced-media-sending

Conversation

@moothz
Copy link
Copy Markdown

@moothz moothz commented Apr 4, 2026

Introduces several enhancements to media handling:

  • GifPlayback: Support for sending MP4 videos that loop as GIFs in the client.
  • Video Stickers: Adds the ability to convert video clips into animated WebP stickers.
  • Transparent Color: Adds a transparentColor parameter to the sticker endpoint, allowing background removal via FFmpeg colorkey filter.

These changes have been in effect in my codebase since december (early acess) - throughly tested.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Performance improvement

Testing

  • Manual testing completed
  • Functionality verified in development environment
  • No breaking changes introduced

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have tested my changes thoroughly
  • Any dependent changes have been merged and published

Summary by Sourcery

Enhance media sending to support GIF-like video playback and richer sticker generation options.

New Features:

  • Allow video messages to be flagged for GIF-like looping playback based on GIF file extensions.
  • Support converting remote MP4 videos into animated WebP stickers when sending stickers via URL.
  • Add an optional transparentColor parameter to sticker sending requests to remove a specific background color during FFmpeg-based video-to-sticker conversion.

Documentation:

  • Document GIF URL support for video messages and their automatic conversion to looping playback.
  • Update sticker API documentation to cover video-based animated stickers and the transparentColor background removal option, with new JSON and cURL examples.

@sourcery-ai
Copy link
Copy Markdown

sourcery-ai bot commented Apr 4, 2026

Reviewer's Guide

Implements enhanced media sending by flagging MP4s sourced from GIF URLs for GIF-like video playback, extending sticker sending to support video-to-animated-WebP conversion with optional background removal via a transparentColor parameter, and updating API docs accordingly.

Sequence diagram for sending animated video sticker with transparent background

sequenceDiagram
    actor Client
    participant API as HTTP_API
    participant Service as sendService
    participant Conv as convertToWebP
    participant VidConv as convertVideoToWebP
    participant FF as ffmpeg
    participant WA as WhatsApp_Server

    Client->>API: POST /send/sticker {number, sticker URL, transparentColor}
    API->>Service: SendSticker(StickerStruct)
    alt Sticker is URL
        Service->>Conv: convertToWebP(stickerURL, transparentColor)
        Conv->>API: http.Get(stickerURL)
        API-->>Conv: Response body bytes
        Conv->>Conv: Detect MIME type
        alt MIME is video/mp4
            Conv->>VidConv: convertVideoToWebP(data, transparentColor)
            VidConv->>FF: run ffmpeg with colorkey and base filters
            FF-->>VidConv: animated WebP file
            VidConv-->>Conv: webpData []byte
        else MIME is image (jpeg/png/jpg)
            Conv->>Conv: Decode image and encode to WebP
        end
        Conv-->>Service: webpData []byte
    end
    Service->>WA: Send sticker message with WebP media
    WA-->>Service: Message ID
    Service-->>API: MessageSendStruct
    API-->>Client: 200 OK with message info
Loading

Class diagram for enhanced media sending and sticker conversion

classDiagram
    class sendService {
        +sendMediaFileWithRetry(data MediaStruct, fileData []byte, instance instance_model.Instance) MessageSendStruct
        +sendMediaUrlWithRetry(data MediaStruct, instance instance_model.Instance) MessageSendStruct
        +SendSticker(data StickerStruct, instance instance_model.Instance) MessageSendStruct
    }

    class StickerStruct {
        +string Number
        +string Sticker
        +string Id
        +int32 Delay
        +[]string MentionedJID
        +bool MentionAll
        +*bool FormatJid
        +string TransparentColor
        +QuotedStruct Quoted
    }

    class MediaStruct {
        +string Filename
        +string Url
        +string Caption
    }

    class waE2EVideoMessage {
        +string Caption
        +string URL
        +string DirectPath
        +string Mimetype
        +[]byte FileSHA256
        +uint64 FileLength
        +[]byte FileEncSHA256
        +bool GifPlayback
    }

    class convertToWebP {
        +convertToWebP(imageDataURL string, transparentColor string) []byte
    }

    class convertVideoToWebP {
        +convertVideoToWebP(inputData []byte, transparentColor string) []byte
    }

    sendService --> StickerStruct : uses
    sendService --> MediaStruct : uses
    sendService --> waE2EVideoMessage : builds
    sendService --> convertToWebP : calls
    convertToWebP --> convertVideoToWebP : calls

    class ffmpegProcess {
        +string inputPath
        +string outputPath
        +string filters
    }

    convertVideoToWebP --> ffmpegProcess : spawns
Loading

File-Level Changes

Change Details Files
Add GifPlayback handling for videos derived from GIF filenames/URLs in media send flows.
  • Derive an isGif flag by checking .gif suffix on media filename or URL for video type.
  • Populate GifPlayback on VideoMessage payloads in both file-based and URL-based send paths, including newsletter messages.
  • Ensure VideoMessage construction remains backward compatible aside from the new flag.
pkg/sendMessage/service/send_service.go
Extend sticker sending to support video sources and optional background color keying via FFmpeg.
  • Augment StickerStruct with a transparentColor field exposed on the sticker send API.
  • Replace the simple URL-to-WebP image converter with a format-aware pipeline that detects MIME type and branches between passthrough WebP, FFmpeg-based MP4-to-animated-WebP conversion, and image-to-WebP re-encoding.
  • Implement convertVideoToWebP using temporary files and FFmpeg filters including fps, scaling, padding, and optional colorkey when transparentColor is provided.
  • Wire SendSticker to pass transparentColor through to the WebP conversion function.
pkg/sendMessage/service/send_service.go
Update public API documentation to describe GIF-like video sending and video sticker support with transparentColor.
  • Expand accepted video formats to include GIF URLs that are converted to MP4 with GifPlayback semantics.
  • Clarify that sticker URLs may be images or videos and document the transparentColor hex parameter for background removal.
  • Add static vs animated sticker cURL examples, including one showing transparentColor usage.
docs/wiki/guias-api/api-messages.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Copy Markdown

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 security issue, 3 other issues, and left some high level feedback:

Security issues:

  • Discovered a potential authorization token provided in a curl command header, which could compromise the curl accessed resource. (link)

General comments:

  • The isGif detection for GifPlayback currently relies only on .gif suffix in Filename/Url; consider basing this on the actual content type (e.g., via existing upload metadata or MIME sniffing) to avoid misclassification when URLs or filenames do not reflect the true media type.
  • In convertVideoToWebP/convertToWebP, the transparentColor parameter is passed directly into the colorkey filter after only stripping #; adding validation (length, hex chars, optional alpha) and normalizing to the exact format FFmpeg expects would make behavior more predictable and avoid subtle filter failures.
  • The http.Get in convertToWebP is a bare call with no timeout or context; using a shared http.Client with sensible timeouts (or a context-aware request) would make remote sticker/video conversion more robust against slow or hanging endpoints.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The `isGif` detection for `GifPlayback` currently relies only on `.gif` suffix in `Filename`/`Url`; consider basing this on the actual content type (e.g., via existing upload metadata or MIME sniffing) to avoid misclassification when URLs or filenames do not reflect the true media type.
- In `convertVideoToWebP`/`convertToWebP`, the `transparentColor` parameter is passed directly into the `colorkey` filter after only stripping `#`; adding validation (length, hex chars, optional alpha) and normalizing to the exact format FFmpeg expects would make behavior more predictable and avoid subtle filter failures.
- The `http.Get` in `convertToWebP` is a bare call with no timeout or context; using a shared `http.Client` with sensible timeouts (or a context-aware request) would make remote sticker/video conversion more robust against slow or hanging endpoints.

## Individual Comments

### Comment 1
<location path="pkg/sendMessage/service/send_service.go" line_range="1425-1434" />
<code_context>
+func convertVideoToWebP(inputData []byte, transparentColor string) ([]byte, error) {
</code_context>
<issue_to_address>
**issue (bug_risk):** Validate `transparentColor` before using it in the `colorkey` filter to avoid ffmpeg failures.

Currently any `transparentColor` is interpolated directly into `colorkey=0x%s:0.1:0.0`. If the value isn’t a valid hex color (length or characters), ffmpeg will likely fail at runtime. Please validate the cleaned hex (e.g., only `[0-9a-fA-F]`, length 6/8) and either normalize or return a clear error instead of relying on ffmpeg’s failure message.
</issue_to_address>

### Comment 2
<location path="pkg/sendMessage/service/send_service.go" line_range="1493-1499" />
<code_context>
+
+	if mime.Is("image/webp") {
+		return data, nil
+	} else if mime.Is("video/mp4") {
+		return convertVideoToWebP(data, transparentColor)
+	} else if mime.Is("image/jpeg") || mime.Is("image/png") || mime.Is("image/jpg") {
</code_context>
<issue_to_address>
**suggestion:** Consider handling additional common formats (e.g. GIF) or documenting the new "unsupported format" behavior.

This change narrows supported formats to WebP, MP4, and a few image types, and now returns "unsupported format" for everything else. For sticker URLs served as `image/gif`, this is a behavior change from "try to decode whatever `image.Decode` supports" to an explicit failure. If GIF stickers are expected, consider adding GIF support (e.g., via ffmpeg or `image/gif`) or clearly documenting that GIF and similar formats are not supported so callers can handle that case.

```suggestion
	mime := mimetype.Detect(data)

	// Supported sticker formats:
	//   - image/webp: returned as-is
	//   - video/mp4: converted to WebP
	//   - image/jpeg, image/jpg, image/png, image/gif: decoded and re-encoded as WebP
	// Any other MIME type will result in an "unsupported format" error so callers can handle it explicitly.
	if mime.Is("image/webp") {
		return data, nil
	} else if mime.Is("video/mp4") {
		return convertVideoToWebP(data, transparentColor)
	} else if mime.Is("image/jpeg") || mime.Is("image/png") || mime.Is("image/jpg") || mime.Is("image/gif") {
```
</issue_to_address>

### Comment 3
<location path="pkg/sendMessage/service/send_service.go" line_range="937-939" />
<code_context>
 			}
 			mediaType = "ImageMessage"
 		case "video":
+			isGif := strings.HasSuffix(strings.ToLower(data.Filename), ".gif") || strings.HasSuffix(strings.ToLower(data.Url), ".gif")
 			if isNewsletter {
 				media = &waE2E.Message{VideoMessage: &waE2E.VideoMessage{
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Deriving `GifPlayback` from filename/URL extension may be fragile; consider using a more reliable signal.

`isGif` is derived only from `.gif` suffixes on `data.Filename`/`data.Url`, which can fail when filenames are absent, URLs contain query strings, or the extension doesn’t match the actual content type. Since you already have `mimeType` (or can use upload metadata), prefer inferring `GifPlayback` from a trusted MIME/content-type source rather than the path suffix.

```suggestion
		case "video":
			lowerMimeType := strings.ToLower(mimeType)
			isGif := strings.HasPrefix(lowerMimeType, "image/gif") || strings.HasPrefix(lowerMimeType, "video/gif")
			if isNewsletter {
```
</issue_to_address>

### Comment 4
<location path="docs/wiki/guias-api/api-messages.md" line_range="380-382" />
<code_context>
SUA-CHAVE-API
</code_context>
<issue_to_address>
**security (curl-auth-header):** Discovered a potential authorization token provided in a curl command header, which could compromise the curl accessed resource.

*Source: gitleaks*
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment on lines +1425 to +1434
func convertVideoToWebP(inputData []byte, transparentColor string) ([]byte, error) {
tmpInput, err := os.CreateTemp("", "sticker-input-*.mp4")
if err != nil {
return nil, fmt.Errorf("failed to create temp file: %v", err)
}
defer os.Remove(tmpInput.Name())

if _, err := tmpInput.Write(inputData); err != nil {
tmpInput.Close()
return nil, fmt.Errorf("failed to write to temp file: %v", err)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): Validate transparentColor before using it in the colorkey filter to avoid ffmpeg failures.

Currently any transparentColor is interpolated directly into colorkey=0x%s:0.1:0.0. If the value isn’t a valid hex color (length or characters), ffmpeg will likely fail at runtime. Please validate the cleaned hex (e.g., only [0-9a-fA-F], length 6/8) and either normalize or return a clear error instead of relying on ffmpeg’s failure message.

Comment on lines +1493 to +1499
mime := mimetype.Detect(data)

if mime.Is("image/webp") {
return data, nil
} else if mime.Is("video/mp4") {
return convertVideoToWebP(data, transparentColor)
} else if mime.Is("image/jpeg") || mime.Is("image/png") || mime.Is("image/jpg") {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Consider handling additional common formats (e.g. GIF) or documenting the new "unsupported format" behavior.

This change narrows supported formats to WebP, MP4, and a few image types, and now returns "unsupported format" for everything else. For sticker URLs served as image/gif, this is a behavior change from "try to decode whatever image.Decode supports" to an explicit failure. If GIF stickers are expected, consider adding GIF support (e.g., via ffmpeg or image/gif) or clearly documenting that GIF and similar formats are not supported so callers can handle that case.

Suggested change
mime := mimetype.Detect(data)
if mime.Is("image/webp") {
return data, nil
} else if mime.Is("video/mp4") {
return convertVideoToWebP(data, transparentColor)
} else if mime.Is("image/jpeg") || mime.Is("image/png") || mime.Is("image/jpg") {
mime := mimetype.Detect(data)
// Supported sticker formats:
// - image/webp: returned as-is
// - video/mp4: converted to WebP
// - image/jpeg, image/jpg, image/png, image/gif: decoded and re-encoded as WebP
// Any other MIME type will result in an "unsupported format" error so callers can handle it explicitly.
if mime.Is("image/webp") {
return data, nil
} else if mime.Is("video/mp4") {
return convertVideoToWebP(data, transparentColor)
} else if mime.Is("image/jpeg") || mime.Is("image/png") || mime.Is("image/jpg") || mime.Is("image/gif") {

Comment on lines 937 to 939
case "video":
isGif := strings.HasSuffix(strings.ToLower(data.Filename), ".gif") || strings.HasSuffix(strings.ToLower(data.Url), ".gif")
if isNewsletter {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (bug_risk): Deriving GifPlayback from filename/URL extension may be fragile; consider using a more reliable signal.

isGif is derived only from .gif suffixes on data.Filename/data.Url, which can fail when filenames are absent, URLs contain query strings, or the extension doesn’t match the actual content type. Since you already have mimeType (or can use upload metadata), prefer inferring GifPlayback from a trusted MIME/content-type source rather than the path suffix.

Suggested change
case "video":
isGif := strings.HasSuffix(strings.ToLower(data.Filename), ".gif") || strings.HasSuffix(strings.ToLower(data.Url), ".gif")
if isNewsletter {
case "video":
lowerMimeType := strings.ToLower(mimeType)
isGif := strings.HasPrefix(lowerMimeType, "image/gif") || strings.HasPrefix(lowerMimeType, "video/gif")
if isNewsletter {

Comment on lines +380 to +382
curl -X POST http://localhost:4000/send/sticker \
-H "Content-Type: application/json" \
-H "apikey: SUA-CHAVE-API" \
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security (curl-auth-header): Discovered a potential authorization token provided in a curl command header, which could compromise the curl accessed resource.

Source: gitleaks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant