On-upload malware scanning

On-upload malware scanning in Microsoft Defender for Storage

On-upload malware scanning in Defender for Storage automatically scans blobs when you upload or modify them, so you get fast detection of malicious content. This cloud-native, SaaS-based solution uses Microsoft Defender Antivirus to perform comprehensive malware scans. It keeps your storage accounts secure without extra infrastructure or maintenance.

When you integrate on-upload scanning into your storage accounts, you can:

• Prevent malicious uploads: Stop malware from entering your storage environment at the point of upload.
• Simplify security management: Benefit from automatic scanning without deploying or managing agents.
• Enhance compliance: Meet regulatory requirements by ensuring all uploaded data is scanned for malware.

Malware upload is a top threat for cloud storage. Malicious files can enter and spread within an organization through cloud storage services. Defender for Storage provides a built-in solution to mitigate this risk with comprehensive anti-malware capabilities.

Common use cases for on-upload malware scanning

On-upload malware scanning supports a variety of scenarios, including:

• Web applications: Secure user-generated content uploads in web applications such as tax apps, CV upload sites, and receipt uploads.
• Content distribution: Protect assets like images and videos shared at scale through content hubs or Content Delivery Networks (CDNs), which can be common malware distribution points.
• Compliance requirements: Meet regulatory standards, such as NIST (glossary definition), SWIFT, HIPAA, and DORA, by scanning untrusted content, especially for regulated industries.
• Third-party integration: Ensure third-party data, such as content from business partners or contractors, is scanned to prevent security risks.
• Collaborative platforms: Ensure safe collaboration across teams and organizations by scanning shared content.
• Data pipelines: Maintain data integrity in Extract, Transform, Load (ETL) processes by ensuring no malware enters through multiple data sources.
• Machine learning training data: Protect the quality of training data by ensuring data sets are clean and safe, particularly if they contain user-generated content.

Enable on-upload malware scanning

You can enable on-upload malware scanning by using the Azure portal, PowerShell, Infrastructure as Code templates, or the REST API. Before you begin, review the following prerequisites.

Prerequisites

Before you enable on-upload malware scanning, ensure you meet the following prerequisites:

• Permissions: Owner or Contributor role on the subscription or storage account, or specific roles with the permissions required for Defender for Storage.
• Defender for Storage: Enabled on the subscription or individual storage accounts.
• Microsoft.EventGrid provider: Registered in the Azure Subscription.

To enable and configure malware scanning across your subscriptions while retaining detailed control over individual storage accounts, use one of the following methods:

• Azure built-in policy
• Programmatically using Infrastructure as Code templates, including Terraform, Bicep, and ARM templates
• Azure portal
• PowerShell
• Directly with the REST API

When you enable malware scanning, the system automatically creates an Azure Event Grid System Topic resource in the same resource group as the storage account. The malware scanning service uses this Event Grid System Topic resource to listen to blob upload triggers.

For detailed instructions, see Deploy Microsoft Defender for Storage.

Cost control for on-upload malware scanning

Malware scanning is billed per GB scanned. To provide cost predictability, malware scanning supports setting a cap on the amount of GB scanned in a single month per storage account.

The capping mechanism sets a monthly scanning limit, measured in gigabytes (GB), for each storage account. This monthly scanning limit serves as an effective cost control measure. If a predefined scanning limit is reached for a storage account within a single calendar month, the scanning operation automatically halts.

The scanning halt occurs once the threshold is reached, with up to a 20-GB deviation. Files aren't scanned for malware after the monthly scanning limit is reached. The cap resets at the end of every month at midnight UTC. Updating the cap typically takes up to an hour to take effect.

By default, a limit of 10 TB (10,000 GB) is established if no specific capping mechanism is defined.

Customizable on-upload malware scanning filters

On-upload malware scanning supports customizable filters, which you can use to set exclusion rules based on blob path prefixes, suffixes, and blob size. By excluding specific blob paths and types, such as logs or temporary files, you can avoid unnecessary scans and reduce costs.

Configure filters in the storage account's Microsoft Defender for Cloud Settings tab in the Azure portal or through REST API.

Key points:

• Exclusion-based filters: Use up to 24 filter values to exclude blobs from malware scanning.

• OR logic: A blob is excluded if it meets any one of the specified criteria.

Filter types:

• Exclude blobs or containers with prefix: Define a comma-separated list of prefixes.

  • Format: container-name/blob-name. Start with the container name. Don't include the storage account name.

  • To exclude entire containers, use the prefix of container names without a trailing /.

  • To exclude a single container, add a trailing slash / after the container name to avoid excluding other containers with similar prefixes.

• Exclude blobs with suffix: Define a comma-separated list of suffixes.

  • Suffixes match only the end of the blob names.

  • Use for file extensions or blob name endings only.

• Exclude blobs larger than: Specify the maximum size in bytes for blobs to be scanned. Blobs larger than this value are excluded from scanning.

How malware scanning works

The following sections explain how on-upload malware scanning is triggered and how the scanning pipeline processes blobs.

On-upload malware scanning flow

On-upload scans are triggered by any operation that results in a BlobCreated or in a BlobRenamed event, as specified in the Azure Blob Storage as an Event Grid source documentation. These operations include:

• Uploading new blobs: When you add a new blob to a container.
• Overwriting existing blobs: When you replace an existing blob with new content.
• Finalizing changes to blobs: Operations like PutBlockList or FlushWithClose that commit changes to a blob.

Scanning process

When the service detects a blob upload or modification, the scanning process consists of the following stages:

• Event detection: When a BlobCreated or a BlobRenamed event occurs, the malware scanning service detects the change.
• Blob retrieval: The service securely reads the blob content within the same region as your storage account.
• In-memory scanning: The service scans the content in memory by using Microsoft Defender Antivirus with up-to-date malware definitions.
• Result generation: The service generates the scan result and takes appropriate actions based on the findings.
• Content disposal: The service doesn't retain the scanned content and deletes it immediately after scanning.

Throughput and capacity for on-upload malware scanning

On-upload malware scanning has specific throughput and capacity limits to ensure performance and efficiency across large-scale operations. These limits help control the volume of data that the service processes per minute, ensuring a balance between protection and system load.

• Scan throughput rate limit: On-upload malware scanning can process up to 50 GB per minute per storage account. If the rate of blob uploads exceeds this threshold, the system queues the files and attempts to scan them. However, if the upload rate consistently surpasses the limit, the service might not scan some blobs.

Shared aspects with on-demand scanning

The following sections apply to both on-demand and on-upload malware scanning.

• Additional costs for malware scanning: Azure Storage read operations, blob indexing, and Event Grid notifications.
• View and consume malware scan results: Methods such as Blob index tags, Defender for Cloud security alerts, Event Grid events, and Log Analytics.
• Malware remediation automation: Automate actions like blocking, deleting, or moving files based on scan results.
• Supported content and limitations for malware scanning: Covers supported file types, sizes, encryption, and region limitations.
• Malware scanning access and data privacy: Details on how the service accesses and processes your data, including privacy considerations.
• Handling false positives and false negatives: Steps for submitting files for review and creating suppression rules.
• Blob scans and impact on IOPS: Learn how scans trigger further read operations and update blob index tags.

For detailed information on these topics, see the Introduction to malware scanning article.

Best practices and tips

Use the following best practices to improve protection, cost management, and operational response:

• Set cost control caps for storage accounts, particularly those with high upload traffic, to manage and optimize expenses effectively.
• Use Log Analytics to track scan history for compliance and auditing purposes.
• If your use case requires a malware remediation mechanism, consider using either the built-in soft delete malicious blobs feature or setting up automated remediation (for example, quarantine or deletion actions) using Event Grid and Logic Apps. For detailed setup guidance, see Set up remediation in malware scanning.

Related content

• Introduction to malware scanning
• On-demand malware scanning