Collection::unique() fails to provide all unique values, or incorrectly provides duplicates

[jmarble]

Laravel Version

12.35.1

PHP Version

8.4.13

Database Driver & Version

No response

Description

Collection::unique() and Collection::duplicates() produce incorrect results when working with string data containing numeric-looking values. The methods either lose unique values or fail to remove duplicates.

This is a correctness bug causing data loss in production applications. The workaround (uniqueStrict()/duplicatesStrict()) is correct but 50-200x slower, making it unusable for large datasets.

Real-world impact:

• CRM contact deduplication (phone numbers from multiple sources)
• Product SKU processing (CSV imports with numeric/variant SKUs)
• API request tracking (finding unique failed requests)
• Duplicate detection produces false positives

Root cause: PHP's array_unique() with SORT_REGULAR uses loose comparison for numeric strings ('+15015551234' == '15015551234' → true) and an unstable sort with adjacent-only comparison, causing duplicates separated during sort to never be compared.

Related:
Previous PR attempts to provide a performant solution to the issue were closed:

• [12.x] Add uniqueStrings() method to collections #57517
• [12.x] Add duplicateStrings() method to collections #57520
• Optimize Collection unique() strict mode for string-only collections #57501

PHP Bug Report:
I've submitted a bug report and PR to correct the behavior of SORT_REGULAR, the root cause of the issue.

• array_unique() with SORT_REGULAR returns duplicate values php/php-src#20262
• Fix SORT_REGULAR transitivity for mixed types php/php-src#20315

Steps To Reproduce

Example 1: Phone Numbers (when collecting unique variations)

use Illuminate\Support\Collection;

$phones = collect([
    '9495551234',      // Local format
    '19495551234',     // With country code  
    '+19495551234',    // International format
    '949-555-1234',    // Formatted with dashes
    '9495551234',      // Duplicate of first
]);

$unique = $phones->unique();

echo "Expected: 4 unique phone numbers\n";
echo "Actual: " . $unique->count() . " items\n";
print_r($unique->values()->all());

Expected: 4 unique values ['9495551234', '19495551234', '+19495551234', '949-555-1234']

Actual: 6 items with:

• Lost: '+19495551234' (incorrectly treated as duplicate of '19495551234')
• Kept: '9495551234' appears twice (duplicate not removed)

---

Example 2: Unit Numbers

$units = collect(['5', '10', '5', '3A', '5', '5']);
$unique = $units->unique();

echo "Expected: 3 unique values ['5', '10', '3A']\n";
echo "Actual: " . $unique->count() . " items\n";
print_r($unique->values()->all());

Expected: 3 unique values ['5', '10', '3A']

Actual: 4 items ['5', '10', '3A', '5'] (duplicate '5' not removed)

---

Example 3: Duplicate Detection False Positives

$contacts = collect([
    '5015551234',      // First entry
    '15015551234',     // Different format (unique)
    '+15015551234',    // Different format (unique)
    '5015551234',      // Actual duplicate of first
]);

$duplicates = $contacts->duplicates();

echo "Expected: 1 duplicate at index 3\n";
echo "Actual: " . $duplicates->count() . " duplicates\n";
print_r($duplicates->all());

Expected: 1 duplicate [3 => '5015551234']

Actual: 2 items with false positive [2 => '+15015551234', 3 => '5015551234']

[adityaanurag0219]

I was able to reproduce this issue locally on:

Laravel Version: 12.35.1

PHP Version: 8.4.13

Confirming that Collection::unique() and Collection::duplicates() both incorrectly treat numeric-like strings such as '+19495551234' and '19495551234' as duplicates.

This matches the behavior described in PHP core issue php/php-src#20262
, which is the underlying cause. Once that fix is merged upstream, Laravel’s unique() should behave correctly again without any changes in framework code.

I’ve validated this with a minimal test similar to:

$phones = collect(['19495551234', '+19495551234']);
var_dump($phones->unique()->all());

Output currently excludes '+19495551234' as expected from PHP’s loose comparison behavior.

This confirms it’s a PHP engine issue, not a Laravel bug.

[jmarble]

This confirms it’s a PHP engine issue, not a Laravel bug.

I think Laravel is better than that! Let's provide an elegant fix -- there's no guarantee my PR with PHP will be accepted anytime soon. Besides that, I've already demonstrated methods and provided multiple fully tested BC PRs that provide a solution to this bug.

It noone wants a new significantly more performant method, take this PR -- just be sure to use uniqueStrict().
#57501

[adityaanurag0219]

Thanks @jmarble , that’s a great point.

I completely agree that Laravel could provide an optional workaround while PHP finalizes its upstream fix. Your earlier PRs (like #57501) already demonstrate solid approaches for handling this within the framework itself.

From my side, I wanted to confirm the reproducibility and clarify that the core cause is indeed PHP’s loose numeric comparison, so maintainers have a clear picture of both layers i.e. framework and language.

I’ll continue tracking both this issue and the PHP PR, and will be happy to help test or validate whichever direction the maintainers choose to take.

[macropay-solutions]

In case this will not be handled by Laravel, our free lib crud v 5.0.7 contains a backward compatible feature fix for the case when Eloquent Collection is pluck-ed into a Support Collection on which unique is applied.

Example:

 User::query()->limit(100)->pluck('street_number')->unique()->toArray();

Note that \collect($array)->unique() is NOT covered!

@jmarble thank you very much for inspiring us.

[macropay-solutions]

@jmarble We think you can stop now with trying to fix this in laravel.

[jmarble]

I'm not sure why it's appropriate to convert this bug report into a discussion. As a discussion, the community is more likely to lose focus on this.

One of the greatest things about Laravel, and using a framework in general, is that we're able to very easily implement solutions to non-trivial issues, or like in this case, bugs within the PHP core. This is not unprecedented, take this lovely commit by @taylorotwell to work around a PHP bug in the MailServiceProvider.

The issue as it exists in Laravel was only introduced 4 years ago. Prior to that, the unique() method was correct, albeit slow.

The PRs I introduced for the new uniqueStrings() and duplicateStrings() methods would at least provide a performant workaround for simple collections of scalar types.

I also provided a PR that dramatically improves the performance of unique() when used with "strict" comparison for scalar types.

While I'm sure there is a performant userland solution for complex types, I'm not sure @taylorotwell wants to go down that road.