adversarial attack

Spy GANs : using adversarial watermarks to send secret messages

By dillon niederhut under adversarial attack adversarial watermarking nlp steganography

In the recent posts where we have been discussing data poisoning, we have mostly been focused on one of two things:

  1. an availability attack, where we degrade the accuracy of a model if it gets trained on any data that we generated; or,
  2. a backdoor attack, where the model performance …

When reality is your adversary: failure modes of image recognition

By dillon niederhut under computer vision object recognition adversarial attack

In the typical machine learning threat model, there is some person or company who using machine learning to accomplish a task, and there is some other person or company (the adversary) who wants to disrupt that task. Maybe the task is authentication, maybe the method is identity recognition based on …

Is it illegal to hack a machine learning model?

By dillon niederhut under adversarial attack data poisoning computer fraud and abuse act legal risk

Maybe. It depends a little bit on how you do it and a lot of bit on why.

I am not a lawyer, and this blog post should not be construed as legal advice. But other people who are lawyers or judges have written about this, so we can review …

We're not so different, you and I -- adversarial attacks are poisonous

By dillon niederhut under computer vision adversarial attack data poisoning

I spent a lot of time thinking about the title for this post. Way more than usual! So I hope you'll indulge me in quickly sharing two runners up:

  1. The real data posions were the adversarial examples we found along the way
  2. Your case and my case are the same …

Evading real-time detection with an adversarial t-shirt

By dillon niederhut under computer vision object recognition adversarial attack adversarial shirt evasion attack

In the last blog post, we saw that a large carboard cutout with a distinctive, printed design could help a person evade detection from automated surveillance systems. As we noted, this attack had a few drawbacks -- largely, that the design needed to be held in front of the person's body …

Evading CCTV cameras with adversarial patches

By dillon niederhut under computer vision object recognition adversarial attack adversarial patch evasion attack

In our last blog post, we looked at a paper that used a small sticker (a "patch") to make any object appear to be a toaster to image recognition models. This is known as a misclassification attack -- the model still recognizes that there is an object there, but fails to …

Fooling AI in real life with adversarial patches

By dillon niederhut under computer vision object recognition adversarial attack adversarial patch

In our last blog post, we talked about how small perturbations in an image can cause an object detection algorithm to misclassify it. This can be a useful and sneaky way to disguise the contents of an image in scenarios where you have taken a digital photograph, and have the …

What is adversarial machine learning?

By dillon niederhut under computer vision object recognition adversarial attack

If you work in computer security or machine learning, you have probably heard about adversarial attacks on machine learning models and the risks that they pose. If you don't, you might not be aware of something very interesting -- that the big fancy neural networks that companies like Google and Facebook …

A faster way to generate thin plate splines

By dillon niederhut under python numpy object detection adversarial attack performance

In Evading real-time person detectors by adversarial t-shirt1, Xu and coauthors show that the adversarial patch attack described by Thys, Van Ranst, and Goedemé2 is less successful when applied to flexible media like fabric, due to the warping and folding that occurs.

low success rate with adversarial patch from AUTHORS

They propose to remedy this failure …