Privacy-Preserving Signal Processing

Back to Inalds home page

What is Privacy-Preserving Signal Processing?

Privacy-preserving signal processing aims at processing signals that are encrypted. No decryption is applied to the signals (audio, images, user preferences) when they are processed. The algorithms work directly on the encrypted -- and hence protected -- signals. In this way privacy sensitive information is protected against abuse, while still the usual signal processing algorithm can be executed. An illustrative example is this one:

Recognize the face of the person in this encrypted image.

Can we actually do that?

Yes! Read on! And if you want to know more, read this tutorial paper: Encrypted Signal Processing for Privacy Protection.


Multimedia applications are increasingly executed remotely or “in the cloud”. Some multimedia applications pose serious privacy threats for their users as they rely on privacy-sensitive information that can be misused. For example, a service that recommends books to individual customer, needs to know which books the customer has read or bought before. Thus the customer must give up a bit of his/her privacy such that (s)he can use the recommender service. Similar examples hold for movie recommendation and search engines. An other category of "services" which endanger the privacy of an individual is camera surveillance. For instance, the behavior of all pedestrians in a street are monitored or individuals are recognized using face recognition technology, whereas only a fraction of behaviors or faces are (or should be) of real interest to those who installed the camera surveillance system. Hence the privacy of all is violated.

To protect the privacy of users in such scenarios, an emerging paradigm shows that it is attractive and feasible to combine signal processing and cryptography [1]. This emerging synergetic field is called secure signal processing, or privacy-preserving signal processing.

The new and exciting research field of privacy-preserving signal processing aims at making privacy-sensitive data of the user of such multimedia applications inaccessible by means of encryption. Although it is then impossible for the service provider to access directly the content of the encrypted data without the decryption key, the service provider can still process the data under encryption to perform the required task.

The protocols to process the encrypted data are designed by using cryptographic primitives like homomorphic cryptosystems [2] and secure multiparty computation techniques [3]. Typical signal processing operations such as linear transforms can be carried out using homomorphic cryptographic systems. For more complicated signal processing operation, efficient cryptographic protocols are needed. Such protocols include blinding, multi-party computation, and garbled circuits.

We have developed theory and practice for several privacy-sensitive signal processing algorithms, including privacy-preserving face recognition, secure recommendation, and secure clustering [4,5,6]. Neverthess, there are still many challenges in designing privacy in signal processing.

Secure Shape Recognition Demonstration

We have developed a demonstration of privacy-protected signal processing. The demonstration shows secure shape recognition using visual cryptography. Essentially one can consider this demonstration as a prototypical scenario where a client (user) requests a service provider (server) to match a query to a set of items present in the server's data base. The matching is done in a privacy protected fashion. The server will not learn information about the query, not will the server learn the result of the best match(es). In this particular demonstration the user inputs a quesry as a hand drawn shape, and the server matches this query to a set of shapes in the data base, returning the four shapes that match the query best. Follow this link for more information.

The user wishes to request items with a similar shape from the "retailer" (server). The user sketches a shape on the Android app. This image becomes the query.
After entering a key (pin), the app encrypts the query image. The encrypted query image is submitted to the server. The encryption uses shares. In the demonstration the crypto algorithm to create shares is assumed to be asymmetric. Hence the user can submit the public encryption key to the server but keeps the private decryption key. The server therefore cannot decrypt the query image.
The server calculates the similariry between the encrypted query image and its shapes in the database. It uses the public encryption key of the user. The server does not learn the values of the similarity measure, nor does it learn which shape yields the best match. In fact, the result of the mathcing procedure by the server is an encrypted version of one or more (in this case 4) shapes in its database, without the server knowing which shapes are selected as best match.
The server return the (still encrypted) best matches to the user. The user uses the private key to decrypt the response of the server.
The user obtains the best (in this case) 4 matches from the server.

Disclaimer: This demonstration visually illustrates the principles of privacy protected recommendation (in this case: similar shapes recommendation). The processing on the server side is done completely on encrypted data, indeed showing how recommendation can be carried out on encrypted data. However, the crypto algorithm used to create shares is not very strong (not to say: weak). Hence, in this demonstration the server can easily attack the submitted query. The paper Encrypted Signal Processing for Privacy Protection exclusively uses strong public-private key encryption for privacy protected recommendation. The simplicity of the encryption scheme used in the demonstration is not fundamental, it is just to keep the demo understandable and visually attractive.

More Details

For a tutorial overview on Privacy-preserving Signal Processing, read the paper Encrypted Signal Processing for Privacy Protection (IEEE Signal Processing Magazine, January 2013).

I have given several tutorials and invited talks on the subject. The core presentation on Privacy-preserving Signal Processing, Signal Processing in the Dark, or Signal Processing meets Cryptography, can be found here:


[1] Protection and retrieval of encrypted multimedia content: when cryptography meets signal processing, Zekeriya Erkin, Alessandro Piva, Stefan Katzenbeisser, Reginald L. Lagendijk, Jamshid Shokrollahi, Gregory Neven, and Mauro Barni , EURASIP Journal on Information Security, Volume 2007, p.20 (2007).
[2] A survey of homomorphic encryption for nonspecialists , Caroline Fontaine and Fabien Galand. EURASIP Journal on Information Security, Volume 2007 (2007).
[3] Foundations of Cryptography: Volume 2, Basic Applications. Oded Goldreich, Cambridge University Press, New York, NY, USA (2004).
[4] Privacy-Preserving User Clustering in a Social Network, Zekeriya Erkin, Thijs Veugen, Tomas Toft and Reginald L. Lagendijk , IEEE International Workshop on Information Forensics and Security (2009).
[5] Privacy Enhanced Recommender System, Zekeriya Michael Beye, Thijs Veugen and Reginald L. Lagendijk, Thirty-first Symposium on Information Theory in the Benelux, Rotterdam (2010).
[6] Privacy-Preserving Face Recognition, Zekeriya Erkin, Martin Franz, Jorge Guajardo, Stefan Katzenbeisser, Reginald L. Lagendijk, and Tomas Toft , 9th International Symposium on Privacy Enhancing Technologies, August, p.235-253 (2009).

View my stats