Merge

Solution · E-commerce

Product Catalog Deduplication: Entity Resolution for E-Commerce

Managing a product catalog across multiple marketplaces, supplier feeds, and internal systems inevitably leads to duplicates. "iPhone 15 Pro 256GB" from one feed is the same product as "Apple iPhone 15 Pro 256GB Space Black" from another — but your database treats them as separate entries. Multiply that across thousands of SKUs and you have a data quality problem that erodes search relevance, inflates inventory counts, and confuses customers.

Entity resolution solves this by intelligently matching and merging records that refer to the same real-world product, brand, or seller. In this post, we walk through building a product knowledge graph with Merge that automatically deduplicates catalog data from multiple sources.

Dashboard overview

The Problem: Duplicate Products Across Feeds

E-commerce platforms ingest product data from many sources:

  • Manufacturer catalogs with official product names
  • Marketplace scrapers with seller-specific titles
  • Internal teams with abbreviated names and custom SKUs
  • Affiliate feeds with inconsistent formatting

The same product appears as:

Source A Source B Same product?
iPhone 15 Pro 256GB Apple iPhone 15 Pro 256GB Space Black Yes
Galaxy S24 Ultra Samsung Galaxy S24 Ultra 512GB Yes
Sony WH-1000XM5 Sony WH1000XM5 Yes
Levi's Levis / Levi Strauss Yes
Bose QuietComfort Bose QC45 No (different model)

Without entity resolution, each of these creates a separate listing — duplicating inventory, splitting reviews, and fragmenting analytics.

Solution Architecture

We model the e-commerce domain as a knowledge graph with four entity types connected by relationships:

Product --made_by--> Brand
Product --sold_by--> Seller
Product --belongs_to--> Category
Seller --operates_on--> Category

Merge handles the resolution logic: deterministic matching on SKUs, fuzzy matching on product names, and confidence-scored merging with human review for borderline cases.

Step 1: Define Schemas

Each entity type gets a schema that tells Merge which attributes to use for matching and how to weight them.

# Product schema — SKU is deterministic (exact match = guaranteed merge)
curl -X POST https://merge-mdm.com/v1/schemas \
  -H "X-API-Key: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "entity_type": "Product",
    "attributes": [
      {"name": "name", "type": "string", "identity": true, "required": true},
      {"name": "sku", "type": "string"},
      {"name": "category", "type": "string"},
      {"name": "price_range", "type": "string"}
    ]
  }'

# Brand schema
curl -X POST https://merge-mdm.com/v1/schemas \
  -H "X-API-Key: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "entity_type": "Brand",
    "attributes": [
      {"name": "name", "type": "string", "identity": true, "required": true},
      {"name": "country_of_origin", "type": "string"},
      {"name": "website", "type": "string"}
    ]
  }'

Schemas configured in Merge

Step 2: Define Relationships

Relationships connect entity types and enable graph traversal — "show me all products made by this brand, sold by sellers on Amazon."

curl -X POST https://merge-mdm.com/v1/relationships \
  -H "X-API-Key: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"type": "made_by", "from": "Product", "to": "Brand"}'

curl -X POST https://merge-mdm.com/v1/relationships \
  -H "X-API-Key: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"type": "sold_by", "from": "Product", "to": "Seller"}'

curl -X POST https://merge-mdm.com/v1/relationships \
  -H "X-API-Key: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"type": "belongs_to", "from": "Product", "to": "Category"}'

curl -X POST https://merge-mdm.com/v1/relationships \
  -H "X-API-Key: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"type": "operates_on", "from": "Seller", "to": "Category"}'

Step 3: Ingest Product Data

We ingest records from multiple sources. Merge automatically resolves duplicates as records arrive.

# Batch ingest from catalog source A
curl -X POST https://merge-mdm.com/v1/entities/batch \
  -H "X-API-Key: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "entity_type": "Product",
    "records": [
      {
        "source": "catalog-a",
        "attributes": {
          "name": "iPhone 15 Pro 256GB",
          "sku": "APL-IP15P-256",
          "category": "Smartphones",
          "price_range": "$999-$1199"
        }
      },
      {
        "source": "catalog-a",
        "attributes": {
          "name": "Galaxy S24 Ultra",
          "sku": "SAM-S24U-256",
          "category": "Smartphones",
          "price_range": "$1199-$1419"
        }
      },
      {
        "source": "catalog-a",
        "attributes": {
          "name": "Sony WH-1000XM5",
          "sku": "SNY-WH1000XM5",
          "category": "Headphones",
          "price_range": "$299-$399"
        }
      }
    ]
  }'

# Later, records from catalog source B arrive with different naming
curl -X POST https://merge-mdm.com/v1/entities/batch \
  -H "X-API-Key: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "entity_type": "Product",
    "records": [
      {
        "source": "catalog-b",
        "attributes": {
          "name": "Apple iPhone 15 Pro 256GB Space Black",
          "sku": "APL-IP15P-256",
          "category": "Smartphones",
          "price_range": "$999-$1199"
        }
      },
      {
        "source": "catalog-b",
        "attributes": {
          "name": "Samsung Galaxy S24 Ultra 512GB",
          "sku": "SAM-S24U-512",
          "category": "Smartphones",
          "price_range": "$1299-$1419"
        }
      },
      {
        "source": "catalog-b",
        "attributes": {
          "name": "Sony WH1000XM5",
          "sku": "SONY-WH1000XM5",
          "category": "Headphones",
          "price_range": "$299-$399"
        }
      }
    ]
  }'

Entities after ingestion

Step 4: Resolution in Action

Merge processes each incoming record against existing entities using multiple signals:

Auto-Merged (High Confidence ≥ 0.9)

Record A Record B Signal Result
iPhone 15 Pro 256GB Apple iPhone 15 Pro 256GB Space Black Deterministic SKU match (APL-IP15P-256) Auto-merged
Galaxy S24 Ultra Samsung Galaxy S24 Ultra 512GB Fuzzy name (0.91) + category overlap Auto-merged
Nike Nike Exact name + website match Auto-merged
Levi's Levis / Levi Strauss Fuzzy name + website match Auto-merged (3 sources)

Sent to Review (Confidence 0.7–0.9)

Record A Record B Name Similarity Decision
Sony WH-1000XM5 Sony WH1000XM5 0.987 Accepted (same product, hyphen difference)
iPhone 15 Pro 256GB iPhone 15 Pro 256 GB 0.705 Accepted (spacing variation)
Bose QuietComfort Bose QC45 0.838 Rejected (different model generation)
Headphones Smartphones 0.779 Rejected (false positive)

Pending reviews before resolution

Step 5: Human-in-the-Loop Reviews

Merge surfaces borderline matches for human review. The review queue shows the candidate entity, source record, confidence score, and comparison details — making it easy to accept or reject in seconds.

# Accept a review (merge the records)
curl -X POST https://merge-mdm.com/v1/reviews/{review_id}/accept \
  -H "X-API-Key: $API_KEY"

# Reject a review (keep as separate entity)
curl -X POST https://merge-mdm.com/v1/reviews/{review_id}/reject \
  -H "X-API-Key: $API_KEY"

Reviews after resolution

Step 6: Inspect Merged Entities

After resolution, a single golden entity represents each real-world product with full provenance tracking.

# View the merged iPhone entity with all source records
curl https://merge-mdm.com/v1/entities/ent_88ad0f63-3cc0-4352-bd15-de987d96f3e2/sources \
  -H "X-API-Key: $API_KEY"

Response shows 3 sources merged into one entity:

{
  "sources": [
    {
      "source_system": "catalog-a",
      "raw": {"name": "iPhone 15 Pro 256GB", "sku": "APL-IP15P-256"}
    },
    {
      "source_system": "catalog-b",
      "raw": {"name": "Apple iPhone 15 Pro 256GB Space Black", "sku": "APL-IP15P-256"}
    },
    {
      "source_system": "catalog-b",
      "raw": {"name": "iPhone 15 Pro 256 GB", "sku": "IP15PRO-256"}
    }
  ]
}

Merged iPhone entity detail

Merged Sony entity detail

Step 7: Search and Discovery

Merge provides fuzzy search across resolved entities — searching for "iphone 15" returns the single golden entity regardless of which source name variation you use.

curl "https://merge-mdm.com/v1/entities/search?q=iPhone&entity_type=Product" \
  -H "X-API-Key: $API_KEY"

Search interface

Results Summary

Starting from 35 ingested records across multiple sources, Merge resolved them into 31 golden entities with:

Metric Value
Total entity creates 31
Automatic merges 6
Reviews generated 4
Reviews accepted 2
Reviews rejected 2
Final pending reviews 0

The system correctly:

  • Merged products with matching SKUs regardless of name variations
  • Merged brand names across spelling variants (Levi's → Levis → Levi Strauss)
  • Flagged borderline cases like "Sony WH-1000XM5" vs "Sony WH1000XM5" for review
  • Rejected false positives like "Bose QuietComfort" vs "Bose QC45" (different products)

Levi Strauss brand — 3 sources merged

Tips for E-Commerce Entity Resolution

  1. Use SKUs as deterministic matchers. When two records share an exact SKU, they are the same product — no fuzzy logic needed. This catches the majority of duplicates instantly.

  2. Normalize before ingesting. Strip whitespace variations, standardize case, and expand abbreviations at the source level to improve fuzzy match accuracy.

  3. Set conservative auto-merge thresholds. A threshold of 0.9 means only very high-confidence matches merge automatically. False merges are expensive to undo — false non-merges are cheap to review.

  4. Leverage supporting attributes. Category, price range, and brand act as confirmation signals. Two products with the same name in the same category and price range are far more likely to be duplicates.

  5. Separate models from families. "Bose QuietComfort" and "Bose QC45" scored 0.84 similarity but are different products. Use model numbers and generation identifiers as distinct attributes to prevent false merges.

  6. Build the graph. Relationships between products, brands, sellers, and categories enable powerful queries: "find all electronics sold by top-rated sellers" traverses the graph rather than scanning flat tables.

  7. Monitor resolution quality. Track your accept/reject ratio on reviews. A high reject rate means your thresholds need tuning or your data needs better normalization.

Conclusion

Entity resolution transforms messy, duplicated product catalogs into a clean knowledge graph where each real-world product, brand, and seller has a single authoritative representation. Merge handles the heavy lifting — deterministic matching, fuzzy name comparison, confidence scoring, and human-in-the-loop review — so your team focuses on decisions rather than deduplication.

The combination of automatic merging for high-confidence matches and human review for borderline cases gives you both speed and accuracy. Start with your highest-volume entity types (products), add relationships to build the graph, and let the system learn from review decisions to improve over time.

Ready to deduplicate your product catalog? Get started at merge-mdm.com.

Start free All solutions