What a lovely hat

Paper 2026/464

Model Extraction of Convolutional Neural Networks with Max-Pooling

Abstract

Model extraction attacks aim to recover the internal parameters of neural networks through black-box queries. While significant progress has been achieved for fully connected ReLU networks, far less is known about structured architectures such as Convolutional Neural Networks (CNNs), which are widely used in practice. In particular, convolutional layers introduce locality and weight sharing, while max-pooling operations leak only relative activation information, both of which require rethinking and extending existing extraction techniques. In this work, we study the extraction of CNNs combining ReLU activations and max-pooling layers in the soft-label setting. We first demonstrate that max-pooling can be understood as a natural extension of the ReLU non-linear operation, where the attacker only has access to relative information between neurons. The local structure of convolution allows us to overcome this difficulty and reconstruct the underlying convolutional kernel. We also introduce optimizations that take advantage of the specific structure of CNNs: by using receptive-field analysis, we design efficient methods to filter noise and localize critical points. These improvements significantly reduce the computational cost compared to a naive reduction to a large sparse fully connected network. Finally, we validate our methodology experimentally on a compact VGG-style convolutional neural network trained on CIFAR-10. The results demonstrate successful layer-by-layer extraction in practice, accurate localization of critical points, and significant efficiency gains from receptive-field-based localization.