Abstract: Deep neural networks (DNNs) have been regarded as fundamental tools for many disciplines. Meanwhile, they are known for their large-scale parameters, high redundancy in weights, and extensive computing resource consumptions, which pose a tremendous challenge to the deployment in real-time applications or on resource-constrained edge devices.  To cope with this issue, compressing DNNs for accelerating its inference has drawn extensive interest earlier.  In the first part of this talk, we will showcase how vison transformer can be pruned with little performance degradation to meet onboard resource constraints from a large-scale, constrained, multi-objective optimization perspective. EvolutionViT is proven to effectively tradeoff between computational cost and performance under resource constraints, automatically searching for neural architectures while optimizing two conflicting objectives. Neural architecture search (NAS) aims to automate the architecture design process, which typically requires a great deal of domain knowledge and human ingenuity.  Architectures designed with NAS algorithms have outperformed superior manually crafted networks on a variety of tasks.  Nonetheless, the success of NAS is often accompanied by a significant investment in computing resources in order to evaluate numerous candidate architectures during the search process.  Such prohibitive computational expenses would obviously deter many interested practitioners without vast computing resources.  Consequently, as one of the most pressing issues, exploring effective means to speed up the search process has been intensively studied in recent years.  As research in neural architecture search advances, a wide variety of accelerating methods continue to emerge, progressively compressing the computational complexity of NAS.  Starting from low-fidelity approximations, performance predictors, and one-shot NAS, the search cost for NAS has dropped from thousands of GPU days to a few GPU days or even hours.  However, these methods still rely heavily on validated accuracy as a metric to guide architecture search, and they all essentially involve network training, whether for candidate architectures or supernets, which inevitably imposes a sizable computational overhead.  Fortunately, of late, zero-cost proxies have emerged that work to predict the architecture performance at initialization based on some theoretical understanding of deep neural networks. Since only a single forward inference/backward propagation is required, they dramatically slash the search cost of NAS further to the order of minutes or seconds with a single GPU.  In the second part of this talk, we will discuss a fine-grained perturbation-aware term to measure how well the architecture can distinguish between inputs and their perturbed counter parts.  We propose a layer-wise score multiplication approach to combine these two scoring terms, deriving a new proxy, named efficient perturbation-aware distinguishing score (ePADS). Experiments on various NAS spaces and datasets show that ePADS consistently outperforms other zero-cost proxies in terms of both predictive reliability and efficiency.