Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Domains visited get leaked to DDG servers #527

Closed
Tritonio opened this issue Jul 9, 2019 · 96 comments
Closed

Domains visited get leaked to DDG servers #527

Tritonio opened this issue Jul 9, 2019 · 96 comments

Comments

@Tritonio
Copy link

Tritonio commented Jul 9, 2019

private const val faviconBaseUrlFormat = "https://icons.duckduckgo.com/ip3/%s.ico"

This seems to be leaking all(?) the domains that users visit to your servers.

@tagawa
Copy link
Contributor

tagawa commented Jul 11, 2019

Hi @Tritonio and thanks for your feedback. The purpose of the request you observed is to retrieve a website's favicon so that it can be displayed in certain places within the app or on the results page. We use an internal favicon service because it can be complicated to locate a favicon for a website. They can be stored in a variety of locations and in a variety of formats. The service understands these edge cases and simplifies retrieval within our apps and our search engine.

At DuckDuckGo, we do not collect or share personal information. That's our privacy policy in a nutshell. For more detailed information on that, you can checkout our privacy policy at https://DuckDuckGo.com/privacy. The favicon service, as with all our services, adheres to this privacy policy in that the requests are anonymous and do not collect or share any personal information.

If you have further questions, please let me know.

@stefan01
Copy link

Hi @tagawa,
i do trust DDG not using the data in a bad manner, but still i don't get your decision.
The best privacy products are made from companies you don't need to trust using your data in a good way, because they collect no data.
So for a privacy product it is really hard for me to understand why you chose a minimal performance increase (?) over a critical privacy issue (in my opinion).
I think with this decision you are risking a major privacy distrust.

@donfuxx
Copy link
Contributor

donfuxx commented Jul 12, 2019

I understand that there are different favicon standards and that in some cases it can be difficult to locate it.
I believe that those are edge cases though and for a vast majority of websites a simple host/favicon.ico should work.
My suggestion for an increased privacy here is:

  1. Attempt to load favicon directly from website by simply appending /favicon.ico to hostname (should be the most easy and most common case anyway)
  2. In case of no success fallback to use omniscient ddg favicon endpoint

@stefan01
Copy link

Just to understand: Why can't the same algorithm, which is being used in the DDG endpoint, be utilized in the app?
If I understand correctly the only reasons are slightly increase performance, reduced network traffic and easier implementation.
Is that correct?

@masstransithonchkrow
Copy link

Hi there! If DDG reviews this, please use the header data on my web pages to determine what the favicon is. I do not have a default favicon for my site. My DeskThemePacks use PNGs for favicons based on the currently viewed theme's normal cursor.

By looking for favicon.ico where it doesn't exist, it affects both my site's performance and SEO.

@timmc
Copy link

timmc commented Jul 2, 2020

If it was just search results, I wouldn't care.

But this appears to be in use for rendering tabs that you're visiting, regardless of how you got there:

return Uri.parse(resolvedUrl())?.faviconLocation()

(ETA: Or am I misunderstanding how this app is used? I'm not a smartphone user, and only use DDG in a regular browser.)

@tonyxu-io
Copy link

It’s shocking to see how DDG staff responded to this issue. Doesn’t seem to care much about user privacy concerns.

@jeevank
Copy link

jeevank commented Jul 2, 2020

@CDRussell @tagawa This needs to be reopened and looked into. This is a serious privacy issue regardless of your "privacy policy"

@sheerun
Copy link

sheerun commented Jul 2, 2020

This made me really doubt privacy of DuckDuckGo. How it can be private and secure if you hire security experts who allow it to happen since 2019? "it can be displayed in certain places within the app or on the results page"

@davewood
Copy link

davewood commented Jul 2, 2020

Take the code from Firefox iOS or Android-components. We spent a lot of time on these and it is all on device.

https://github.com/mozilla-mobile/android-components

https://github.com/mozilla-mobile/Firefox-iOS 

[https://news.ycombinator.com/item?id=23708166]

@javabeanz
Copy link

privacy should be the whole raison d'etre of DDG. time for a fork DuckDucGone ?

@scabros
Copy link

scabros commented Jul 2, 2020

I don't know why all the fuss... You all know that if you are using his app, you already "trust" that his search service is respecting your privacy, right? C'mon, they are being transparent, as usual.

@tonyxu-io
Copy link

I don't know why all the fuss... You all know that if you are using his app, you already "trust" that his search service is respecting your privacy, right? C'mon, they are being transparent, as usual.

@scabros - Please checkout the reply above: #527 (comment) This is not about using their search service, we all know search provider know exactly what you are searching for. But this particular issue is about using the browser, it's when user directly visit the website from browser address bar or reference link.

@svenssonaxel
Copy link

So essentially, the trade-off is between performance and auditability of privacy. I predict that the user segment DDG appeals to will overwhelmingly prefer the latter.

Yes, we already trust DDG, but only because we have to trust someone and others have proved to be untrustworthy. The issue isn't about whether the user trusts DDG, it's about minimizing the need for trust and maximizing the ability to verify privacy. Please consider reopening this issue.

@eighthave
Copy link

I think this would make this app qualify in F-Droid for the Tracking anti-feature, there is no good reason why every website URL should be sent to DDG servers. Regardless of intent, it does key bits of data, and there are better technical solutions to finding the favicon.

@solomoncaygnuyou
Copy link

I believe reopening and rectifying this design choice would go a long way to preserving the trust that DDG has worked hard to earn in the privacy-oriented community. Even if users trust that DDG is not using this data in any undesirable way, it would provide more peace of mind to users that the data not be sent in the first place.

I am strongly in favor of changing this functionality to no longer leak visited domains.

@holderbaum
Copy link

holderbaum commented Jul 2, 2020

A long time DDG user and advocate here.

First and foremost, thank you for building the most awesome search engine there is right now! 🙂

I must say that I am pretty appalled by this implementation. Yes, I trust DDG with my search data since I have to do searches and DDG always seemed like they actually care about privacy. But at the same time, a privacy aware browser should never send close to all the user activity to it's server, even if it is "only" the domains that are browsed. I think most DDG users could live with a slightly less then optimal display of favicons in order to gain the assurance, that their domain history is indeed private to their device. That latter is literally the main reason, why most people install your browser in the first place.

I don't think this should be a discussion about "privacy policies". You are integrating a piece of code and infrastructure in your product, that has the capabilities to spy on people. It just shouldn't be there in the first place.

You should not keep this issue just closed like this, this is a serious matter and it is obvious from the community reactions that this needs to be discussed. (https://news.ycombinator.com/item?id=23708166)

@black-snow
Copy link

How is this still closed?

@stefanct
Copy link

stefanct commented Jul 2, 2020

This "feature" has been added in March 2018: 03f99c4#diff-63ac5c0d645555fe179e72977d9c1728

And as of now it is still unchanged in the develop branch:

private const val faviconBaseUrlFormat = "https://proxy.duckduckgo.com/ip3/%s.ico"

(I did not look into the control flow to determine when this gets executed at all though.)

@thors
Copy link

thors commented Jul 2, 2020

I don't know why all the fuss... You all know that if you are using his app, you already "trust" that his search service is respecting your privacy, right? C'mon, they are being transparent, as usual.

  1. This is not only about the search service.
  2. Trust is only a last resort, after they did everything reasonably possible to avoid acquiring unnecessary data in the first place (which they apparently don't).

There are plenty of good browsers on the market. This browser had once privacy as a main selling point, but as it looks, not any more.

@RokerHRO
Copy link

RokerHRO commented Jul 2, 2020

@tagawa : Trust is a very brittle thing. It grows slowly and takes years to built it. And only one line of code can shatter it. But instead to react professionally and contritely you made it worse to stamp on the shards to make sure no useful piece of trust will survive. You had one job! :-(

@bluesign
Copy link

bluesign commented Jul 2, 2020

Sorry but this is not enough reason. There is a simple question you should ask to yourself.

  • Would you be ok to use a third party for this with same privacy policy?

@zink-chimaera
Copy link

At DuckDuckGo, we do not collect (...) personal information

Meanwhile DDG literally collecting personal information from my device. 😕

DDG changed security issue: Ignored

@pythoneer
Copy link

pythoneer commented Jul 2, 2020

For those that don't really understand what is happening:

It is really about that code snippet from here:

private const val faviconBaseUrlFormat = "https://icons.duckduckgo.com/ip3/%s.ico"

fun Uri?.faviconLocation(): Uri? {
    val host = this?.host
    if (host.isNullOrBlank()) return null
    return Uri.parse(String.format(faviconBaseUrlFormat, host))
}

The function for retrieving the Favicon is asking for the current host the user is on (e.g. apple.com) and sends it to a service hosted by duckduckgo.com that is saved in the const faviconBaseUrlFormat. %s is replaced with the actual host the user is on and is send to the duckduckgo.com servers. As an example (change it yourself to whatever host you want):

https://icons.duckduckgo.com/ip3/apple.com.ico

I guess they have a cache for already visited hosts etc.

This sends every host the user visits (or the Favicon is requested for) to the duckduckgo.com server. I think it does not really matter what you do or do not do with the data. Its not about trust – its about the minimum you need to trust and this is clearly breaking a lot of peoples perception of what the browser should achieve. As it currently stands, i would consider this even worse of what i imagine googles chrome would do to its users. To make it clear, i don't know exactly what google chrome is doing – its the perception of what i think certain vendors are doing, and i would not expect this from a "privacy first" browser from duckduckgo.com

Tired of being tracked online? We can help.

Please help!

@GarbageHamburger
Copy link

99% of the websites I have seen have either

  1. a <link rel="shortcut icon">, which directly points to the favicon
  2. if not, a /favicon.ico

I really don't see the point of this URL anyhow. It's useful for search results (because you'd have to download every page to parse its favicon otherwise, and your search results are already served directly from DDG so there'd be no point), but using it in tabs is not the right way. I'm willing to say this is just laziness rather than malice, though: rather than writing code to figure out the favicon from the page, they just used whatever they normally used on the search results page.

NOTE: DDG results page uses external-content.duckduckgo.com instead of proxy.duckduckgo.com, and

~$ host proxy.duckduckgo.com
proxy.duckduckgo.com is an alias for external-content.duckduckgo.com.
~$ host icons.duckduckgo.com
icons.duckduckgo.com is an alias for external-content.duckduckgo.com.

They also seem to be caching the most common favicons like #527 (comment) mentioned. i.e. visit "apple" on DDG and the top link has its favicon set as https://duckduckgo.com/assets/icons/favicons/apple.png.

Please reopen and fix.

@lethargosapatheia
Copy link

What I find most disturbing is that fact that they haven't said a word on this topic for a year now.

@GarbageHamburger
Copy link

What I find most disturbing is that fact that they haven't said a word on this topic for a year now.

I would expect them to respond now that the link blew up on HN.

@alex-tee
Copy link

alex-tee commented Jul 2, 2020

At DuckDuckGo, we do not collect [...] personal information.

first post says otherwise.

@jonathanronen
Copy link

We love DDG for privacy, not for favicons. Privacy starts from not collecting data.

@Tritonio
Copy link
Author

Tritonio commented Jul 2, 2020

why not just reuse this purpose-built proxy they already had lying around. Makes perfect sense.

At best this just shows how high a bar DDG has managed to set for itself that its users are holding it to.

Not leaking every single domain a user visits is not a high bar. It's the bar you need to pass not to be considered spyware. FFS why are you people trying to defend an idiotic code decision? They are fixing it, great, I'm really happy for that. That doesn't mean we should trivialize the matter.

is already leaps and bounds ahead of the competition in

Leaking the domains to a server puts you leaps behind your competition and even your non competition since even regular browsers that don't focus on privacy end up being more private than you.

DYOR but this is why I think these bugs are very serious:

40.114.178.124 is the IP that icons.duckduckgo.com resolves to and WHOIS says that it's managed by Microsoft. If this is Azure, which I would take a bet it is, then that means that for the last year or more, Microsoft could, technically tap into that server's memory to see who requested which domain and when. So not only do you need to trust DDG, you also need to trust Microsoft that they didn't do anything bad with the info. Does anyone still remember PRISM? This kind of passive access to user data is exactly what PRISM was about. Although I could be wrong and maybe the actual server is running in DDG's premises and Microsoft is simply acting as a reverse proxy.

@lemiesz
Copy link

lemiesz commented Jul 2, 2020

Seems like even if you could not fix the favicon serving issue, just deleting the favicon and not having any would be a better user experience. Given that putting the favicon up is breaking a core value proposition of your service.

@nilnilnil
Copy link
Contributor

Hi all, CTO of DuckDuckGo here. I thought it might be helpful if I briefly shared some of our internal thinking around this issue so folks can see how we got here and how we plan to move forward.

The logic behind how we’ve been displaying favicons in our apps is a function of how we operate our private search engine. Since we already have an anonymous favicon service through our search engine, using it has a number of benefits: it avoids more requests to known non-anonymous websites that are visited, it's way faster since it runs server side, saves user bandwidth, and the only externally visible attribute is that the app is connecting to DuckDuckGo.com (as the favicon location is actually encrypted in the path in transit). To our team, utilizing this anonymous service we had made for the search engine seemed like an optimal principled choice across a set of criteria.

We want to be clear that at no point was the actual visited domain otherwise exposed. This favicon service is fully anonymous on our end, and URL parameters (like the favicon domain) are encrypted in transit, just like the search engine (with search queries). This is also why when this issue was raised in the past, we decided to keep the solution as is. At no point was this ignored.

However, we understand that there is an alternative method of getting the favicons locally that a lot of folks prefer while still maintaining our privacy standards. We also believe that method is in line with our product vision of "Privacy, simplified.", considering its a somewhat simpler method than the one we had been using.

So, we went ahead today and implemented the change for both Android (#878) and iOS (duckduckgo/iOS#667) that will move this logic onto the client, and we will no longer be using the favicon service in our apps. These changes are currently in the release phase and are rolling out live now.

We really appreciate the feedback and exchange of ideas on this topic, and on ways to further improve the privacy of our products in general.

@calimeroteknik
Copy link

while still maintaining our privacy standards

That's quite a curious wording to talk about a solution that objectively raises privacy to a much higher standard.
Well, it's good to hear that more privacy "still maintains" your privacy standards.

Please remember, my dear DDG: don't be Google.

@resynth1943
Copy link

I'm personally surprised nobody has talked about the fact that DuckDuckGo's search engine has a strange redirection function when you click on a link. I would love to talk about that at some point, because I really don't understand why it's there.

Many thanks,
Synth

@alectrocute
Copy link

alectrocute commented Jul 2, 2020

There's about seven red herrings in this huge thread. A lot of people who suddenly use DDG when it's convenient to complain about it. Right. @Tritonio was the only one whom I saw explain the actual vulnerability here (provided you have a positive bias of trust for DDG, which disclaimer, I do):

40.114.178.124 is the IP that icons.duckduckgo.com resolves to and WHOIS says that it's managed by Microsoft. If this is Azure, which I would take a bet it is, then that means that for the last year or more, Microsoft could, technically tap into that server's memory to see who requested which domain and when. So not only do you need to trust DDG, you also need to trust Microsoft that they didn't do anything bad with the info. Does anyone still remember PRISM? This kind of passive access to user data is exactly what PRISM was about. Although I could be wrong and maybe the actual server is running in DDG's premises and Microsoft is simply acting as a reverse proxy.

Ah, SIGINT. Spooky SIGINT.

This quote points at Microsoft Azure's IP for example's sake, but that doesn't account for the countless ISPs your data travels through, even on a basic DNS level without vendor participation. A nefarious agency could MITM and use this as spyware for a correlation++ attack specifically targeting DDG users. That should terrify all of us because we love privacy, (right?) and when the right adversary arrives – "could" is usually a synonym for "will inevitably".

I fully 100% believe it's innocent, I even feel it makes perfect sense in the mind of a rushed developer. I am happy to see it fixed. Now, with that said, I'd like to propose a solution so the product marketing people can be happy too.

If you want to have this feature locally, make it secure and do the same thing – just bundle a massive SVG sprite sheet with the client (realistically, how big would it be?) and do some sort of table association to pull them up on demand if the hostname has a match. Otherwise, don't make an outbound network request, just return and wait! If you wanted to change the way you do it (and follow more-or-less standard practice), you can rely on site-provided webContent <head> data as others have mentioned.

Anyway, I'll pass the mic off now. I think @nilnilnil and DDG exec team did a great job of fixing the issue. Up and onwards!

@goranmoomin
Copy link

@resynth1943 It's to not expose the referrer so that the site can't know what the user searched.

@tonyxu-io
Copy link

tonyxu-io commented Jul 3, 2020

@resynth1943 It's to not expose the referrer so that the site can't know what the user searched.

This can be prevented by:

<a href="example.com" rel="noreferrer">link</a>

@resynth1943
Copy link

Interesting. Thank you, @pcr910303. Personally, I'd love to see Referer die a death, it's about damn time.

@resynth1943
Copy link

resynth1943 commented Jul 3, 2020

@resynth1943 It's to not expose the referrer so that the site can't know what the user searched.

This can be prevented by:

<a href="example.com" rel="noreferrer">link</a>

That is curious. When I click on a link in the search engine (here's one about ghosts), I get something like this:

https://duckduckgo.com/l/?kh=-1&uddg=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FGhost_(Swedish_band)

<html><head><meta name='referrer' content='origin'></head><body><script language='JavaScript'>window.parent.location.replace("https://en.wikipedia.org/wiki/Ghost_(Swedish_band)");</script><noscript><META http-equiv='refresh' content="0;URL=https://en.wikipedia.org/wiki/Ghost_(Swedish_band)"></noscript></body></html>

Couldn't this be used to track browsing history, too? Technically, query parameters are sent to the server. Wouldn't mind knowing more about this, if they would explain more about it. An interesting thought would be, if the user has JavaScript enabled, why couldn't DuckDuckGo cook up a URL that looks a little more like this:

https://duckduckgo.com/l/#/kh=-1&uddg=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FGhost_(Swedish_band)

See that tiny change? That little hash prevents the URL from being sent to the server! I wonder how that would work with referers, too. Of course, parsing that hash would need a little bit of JavaScript, but if the user has it enabled, then...

Maybe there's a technical reason why they use this redirection service, instead of noreferrer? Perhaps backwards-compatibility?

EDIT: Looks like I was right.


EDIT: Of course, this wouldn't be a problem if they used POST instead of GET, but I'm told Chrome doesn't play nicely with that.

@tagawa
Copy link
Contributor

tagawa commented Jul 3, 2020

@resynth1943 You're right, the redirect is for backwards-compatibility reasons (there's more about it on this help page) but it can be disabled in the search settings, in which case just the referrer meta tag is used. That's set to origin by the way.

Re. POST and GET, that can also be toggled in the same settings page with GET as the default.

@PranavBhattarai
Copy link

DuckDuckGo just went viral but in a very WRONG way.
This is bad + sad.

@waltercool
Copy link

Come on guys, stop trolling on this post. DuckDuckGo already made an statement on this.

Don't like it, fork it. Stop crying and demanding work from others.

@scabros
Copy link

scabros commented Jul 3, 2020

I am absolutely AMAZED on how people OVERREACTED about this. First disclaimer: I have NO connection to DDG, I am just a user. I work as sysadmin and web developer full time since 2004.

I do agree this is a "feature" that could be implemented locally to let the users feel more comfortable with his privacy concerns.

But this, for me, doesn't represent an anti-privacy practice, as a lot of people are complaining. DDG is basically a search engine. In my own phone I have DDG AND Firefox focus, and use the last as my default browser. Either way, again, I think is a good choice to make the requested change to the local implementation.

What really irritates me is the tone some people come here, complaining in a way that is practically accusing DDG being the same as google & Co. Folks, the distance is HUGE, and i think behind all this there is a key point: it's not about if DDG is receiving your data or not (because, hey, they are a search engine, you know, if you don't input anything, the can't search anything for you). The key point here is what they DO with your data. And they ALWAYS were very clear about that. They anonymize it and process it, to have some valuable info that they could use to improve their products. The "favicon incident" from the users perspective, is just an example of another anonymized service for DDG.

And then you have people coming here saying things like this:

  • best privacy products are made from companies you don't need to trust using your data in a good way, because they collect no data
  • Doesn’t seem to care much about user privacy concerns
  • How it can be private and secure if you hire security experts who allow it to happen since 2019?
  • There are plenty of good browsers on the market. This browser had once privacy as a main selling point, but as it looks, not any more.
  • Meanwhile DDG literally collecting personal information from my device
  • As it currently stands, i would consider this even worse of what i imagine googles chrome would do to its users.
  • But closing the issue just like that is a statement. A huge statement. "We dont care about your privacy"
  • I've just de-installed the Duckduckgo app and also won't be using their search engine anymore. Trust ist lost. Their CEO can put his statement where the sun doesn't shine.
  • And, sorry, but you cannot argue that a company that cares about privacy seriously adds an online service that tracks (and can collect) user's behavior only "by accident".

All this people, use android? or they have hacked their phones and installed librem? could they all do a better job competing against google?
the error was there and it needed to be fixed, it required doing some pressure as his user base, i get it... but this?
"the internet" is full of people very very very bored, and always ready to point out the mistakes of others, as if they did everything right... but they cant even just point an issue in an open, free and transparent service (there are not many out there...), manage his own anger, fears and frustration and being polite to keep asking for it.

To ALL DDG staff, thank you for being there, thank you for giving us an alternative to G, thank you for listening, thank you for being open and transparent and thank you for being condescendent even with those that dont deserve it (because you know there are many that do).
(PS: english is not my first language, I hope you understand )

@resynth1943
Copy link

Come on guys, stop trolling on this post. DuckDuckGo already made an statement on this.

Don't like it, fork it. Stop crying and demanding work from others.

Who's trolling?

@resynth1943
Copy link

@scabros that's fair enough, dude, and I agree with that. People were rude, malicious and forming egregious remarks about the company that just made a mistake.

I can see people have already down voted your post. I've got 31 down votes on a rational opinion, so good luck communicating with the children here.

DuckDuckGo really isn't the second Google.

Far from it.

And that's why I use it.

@tillydray
Copy link

tillydray commented Jul 3, 2020

Unfortunately this isn’t the only “mistake” DDG has made: DuckDuckGo's privacy abuses-- current, historic, and by proxy. People may be overreacting to this post if this post represented a single mistake, but given the context of the post I linked, I suspect most haven’t reacted nearly strongly enough.

Edit: link to hacker news instead of directly to the article

@us31t
Copy link

us31t commented Jul 3, 2020

First of all - I am a big fan of DDG

  • I promote it whenever I can
  • I use it anywhere possible
  • This comment is meant to help the DDG team to understand this PR issue. Even tough - we (OS_Community) are very concerned that this explanation is necessary in the first place! We normally do not rise up from our cellars - because we code. But now where the realms of management and C0D3 collide again, and the management does not show any signs of understanding - we have to take action.

This is why I will now explain, (again) what the problem with your answer is and what it indicates.

Hi all, CTO of DuckDuckGo here. I thought it might be helpful if I briefly shared some of our internal thinking around this issue so folks can see how we got here and how we plan to move forward.

Your 'internal thinking' is what has raised this issue in public! It is basically the bare reason for our concerns! Because it shows, that you do not even understand our privacy argument! It is not about the bug, it is your action and the, like @calimeroteknik puts it, "curios wording", that makes it appear, like you have not understood our critic at all.

Time of Events

  1. @Tritonio found a curios line of code that sends unnecessary requests to one of your backends
  2. @tagawa said: "The purpose of the request you observed is to retrieve a website's favicon so that it can be displayed in certain places within the app or on the results page"
    https://github.com/duckduckgo/Android/pull/878/files#diff-63ac5c0d645555fe179e72977d9c1728
  3. Then @stefan01 and later we said, please stop this! There is no reason to send data in the first place! A big part of privacy focused software is, to avoid to creating unnecessary data in the first place. It is not necessary! Please do not send our browser history or any unnecessary request, to any server! No matter how "secure" this server is. Please load the favicon locally, we love you and we do not like heartbreaks!
    In the worst case - security is only a illusion - only the hacker with root access knows, what is really going on and maybe like @Tritonio indicates Microsoft.
  4. DGG show no real understanding and argues in a "curios way" - and now you @nilnilnil continue

Bugs can happen! Bad decisions can happen! But arguing with "trust us, it anonymous", it "saves user badwidth", and its "the only visible attribute", on top of that "we already have a anonymous [...] service"! It is pure poison for trust, because it indicates, you entirely missed the point! Either on purpose or not. Both would be alarming!

The logic behind how we’ve been displaying favicons in our apps is a function of how we operate our private search engine. Since we already have an anonymous favicon service through our search engine, using it has a number of benefits: it avoids more requests to known non-anonymous websites that are visited, it's way faster since it runs server side, saves user bandwidth, and the only externally visible attribute is that the app is connecting to DuckDuckGo.com (as the favicon location is actually encrypted in the path in transit). To our team, utilizing this anonymous service we had made for the search engine seemed like an optimal principled choice across a set of criteria.

Its not that we do not understand your reasoning or the reasoning of @tagawa. The problem is your reasoning, seems to entirely miss our point!

Now you continue - as CTO! This indicates no real insight at all!

We want to be clear that at no point was the actual visited domain otherwise exposed. This favicon service is fully anonymous on our end, and URL parameters (like the favicon domain) are encrypted in transit, just like the search engine (with search queries). This is also why when this issue was raised in the past, we decided to keep the solution as is. At no point was this ignored.

We said, please do not mix the search engine code and your browser code! They should not relate on each other if not necessary (for example if the user actually wants to do a search on DDG).
Arguing "we already have a anonymous [...] service" is very hard to understand from a technical point of view. It might make sense for a CFO, but not for nerds or hackers!

However, we understand that there is an alternative method of getting the favicons locally that a lot of folks prefer while still maintaining our privacy standards. We also believe that method is in line with our product vision of "Privacy, simplified.", considering its a somewhat simpler method than the one we had been using.

Like @calimeroteknik already said: Luckily higher security standards meet your standard! -_-

So, we went ahead today and implemented the change for both Android (#878) and iOS (duckduckgo/iOS#667) that will move this logic onto the client, and we will no longer be using the favicon service in our apps. These changes are currently in the release phase and are rolling out live now.

You only took this action after enormous public pressure, there was no sign of insight or internal intention to resolve this! This is why you have been compared with BigCorp, because they act like this. (Or like pointed out by some people, are even doing a better job than you, handling these kinds of issues)

[LARP MODE ON]
[Loading Game Data...]
You currently have (3) quests!
[(1) Closed Quest] Even under enormous public pressure, do not remove the criticized code, before at least one year has past!
[(1) Open Quest] Understand the whole point, and communicate it in a acceptable way.
[(1) Bonus Quest] Build trust back by transparent actions in the future, which reflect that you understood what we said
[LARP MODE OFF]

We really appreciate the feedback and exchange of ideas on this topic, and on ways to further improve the privacy of our products in general.

You are welcome! I hope this helps!

@calimeroteknik
Copy link

We really appreciate the feedback and exchange of ideas on this topic, and on ways to further improve the privacy of our products in general.

I overlooked the implications of this. It's much more interesting than it seems.

Consequence 1:

It means that DDG has started a game of

the privacy defects we introduce and you don't notice, we can keep

…with the general public.
The dynamics of this game can be summed up to "we keep hiding things around, and you keep looking for them".

That's immoral, but I believe it's legal and therefore legally allowed to do this. (after all, DDG users are not paying customers)
Additionally, it's a private company: a subpoena can compel it to lie to the public and to its customers.

Consequence 2:

The current evidence unambiguously proves that DDG is commiting at least one of:

  • incompetence concerning basic privacy notions ;
  • basic malevolence, motivated by financial profit and potentially by government pressure.

A real plight, if you want to get away with only PR!

I think that, in order of preference for the public, your options are:
1st. Admit DDG was not respecting privacy in places where it could have, and that the team attempted to not admit it;
2nd. Keep silent, and not do that sort of thing again, so you don't get busted for such practices in the future;
3rd. What you've been doing: say "hey, it's not THAT bad"⁽¹⁾ and "we appreciate the feedback"⁽²⁾.

(1) This is a pretty effective psychological strategy, because it induces self-disgust in people who are not trained to fight it internally. Studying WW2 propaganda helps with understanding that one.
(2) Following it with such benevolent-looking words, like this, is pretty efficient at maximizing your chances at people getting tired of arguing. Unfortunately, that can backfire, which is happening right here.

My sharp words should not hide my encouragements towards duckduckgo fighting its innate corporate nature.
I never took the time to write such a reply to Google, for instance, because I think it's a lost cause whereas DDG isn't, yet.

@The-Compiler
Copy link

This is why you have been compared with BigCorp, because they act like this. (Or like pointed out by some people, are even doing a better job than you, handling these kinds of issues)

FWIW Google has a service used in their Chromium browser (and also used by Firefox and others) for clients to find out whether a given host is malicious (e.g. a known phishing page). Their API can be used in a way in which hostnames are not sent to Google, and even Google Chrome does things that way. (Then again, there might be other features in Chromium sending hostnames to Google, such as the omnibox autocompletion)

@waltercool
Copy link

Who's trolling?

Everyone who keeps posting here, DDG already received a PR and merged at the codebase #878

Is non-sense to keep posting "furious" and "frustrated" comments here. Again, if people dislike DDG Android app in any sense, don't use it, or just fork it.

The whole idea of OpenSource isn't treating developers as slaves who fulfill all demands of everyone, or to make threats at reddit or any other websites (posted bit above) full of conspiracy stuff and making claims of things they aren't true. This kind of behaviors of developers being kind of slaves from users must STOP.

@calimeroteknik
Copy link

@waltercool the discussion here is not about the software.
It is about how DuckDuckGo, the company, responds when privacy problems are pointed out.

Admitting it was a mistake would be a start as far as this discussion goes, and that hasn't happened as far as I can tell.

@waltercool
Copy link

@waltercool the discussion here is not about the software.
It is about how DuckDuckGo, the company, responds when privacy problems are pointed out.

Admitting it was a mistake would be a start as far as this discussion goes, and that hasn't happened as far as I can tell.

I do understand your concern, but please look the "issue" and "official response". Everything else happened after the ticket was closed, basically someone said: "Hey look this code you are leaking all url from users!", and they said "ehhm, no".

A good way to report "a software issue" (because the original claim IS a "code" claim), would be creating a PR proposing a replacement of the DDG favicon server. Companies do opensource not only to expose their code to others, but for community to contribute bugs with change proposals. Reporting a bug doesn't solve anything, proposing solutions does.

And following this, a claim related to "privacy concerns" and not "a code issue" happened at issue #877, which the reported posted a change request at Asana platform, and they accepted it.

@javabeanz
Copy link

favicons can even be weaponized : https://github.com/devanshbatham/FavFreak

@ghost
Copy link

ghost commented Jul 4, 2020

DDG is no difference to google...no privacy at all #my opinion though

@resynth1943
Copy link

resynth1943 commented Jul 5, 2020

I love how GitHub seems to be removing any and all functions that users can use to report abuse. Especially when it's a whole page of spam about video games (?) that's irrelevant to the project.

Nevertheless, I hope DuckDuckGo can learn from the more fruitful responses, one of which has been authored by @calimeroteknik. I think DuckDuckGo's official response was quite... flattering to themselves, and their project. I'll seek to be more wary of products such as this in the future.

@nin0-dev
Copy link

Mates... you gotta do something. At least Brave admitted their mistake and pledged to never inject ref codes on crypto websites. And you, you send our history to DDG servers. And you dare to close the issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests