Probabilistic Safe WCET Estimation for Weakly Hard Real-Time Systems at Design Stages

in ACM Transactions on Software Engineering and Methodology (2023)

Weakly hard real-time systems can, to some degree, tolerate deadline misses, but their schedulability still needs to be analyzed to ensure their quality of service. Such analysis usually occurs at early ... [more ▼]

Weakly hard real-time systems can, to some degree, tolerate deadline misses, but their schedulability still needs to be analyzed to ensure their quality of service. Such analysis usually occurs at early design stages to provide implementation guidelines to engineers so that they can make better design decisions. Estimating worst-case execution times (WCET) is a key input to schedulability analysis. However, early on during system design, estimating WCET values is challenging and engineers usually determine them as plausible ranges based on their domain knowledge. Our approach aims at finding restricted, safe WCET sub-ranges given a set of ranges initially estimated by experts in the context of weakly hard real-time systems. To this end, we leverage (1) multi-objective search aiming at maximizing the violation of weakly hard constraints in order to find worst-case scheduling scenarios and (2) polynomial logistic regression to infer safe WCET ranges with a probabilistic interpretation. We evaluated our approach by applying it to an industrial system in the satellite domain and several realistic synthetic systems. The results indicate that our approach significantly outperforms a baseline relying on random search without learning, and estimates safe WCET ranges with a high degree of confidence in practical time (< 23h). [less ▲]

A Machine Learning Approach for Automated Filling of Categorical Fields in Data Entry Forms

in ACM Transactions on Software Engineering and Methodology (2023), 32(2), 471-4740

Users frequently interact with software systems through data entry forms. However, form filling is time-consuming and error-prone. Although several techniques have been proposed to auto-complete or pre ... [more ▼]

Users frequently interact with software systems through data entry forms. However, form filling is time-consuming and error-prone. Although several techniques have been proposed to auto-complete or pre-fill fields in the forms, they provide limited support to help users fill categorical fields, i.e., fields that require users to choose the right value among a large set of options. In this paper, we propose LAFF, a learning-based automated approach for filling categorical fields in data entry forms. LAFF first builds Bayesian Network models by learning field dependencies from a set of historical input instances, representing the values of the fields that have been filled in the past. To improve its learning ability, LAFF uses local modeling to effectively mine the local dependencies of fields in a cluster of input instances. During the form filling phase, LAFF uses such models to predict possible values of a target field, based on the values in the already-filled fields of the form and their dependencies; the predicted values (endorsed based on field dependencies and prediction confidence) are then provided to the end-user as a list of suggestions. We evaluated LAFF by assessing its effectiveness and efficiency in form filling on two datasets, one of them proprietary from the banking domain. Experimental results show that LAFF is able to provide accurate suggestions with a Mean Reciprocal Rank value above 0.73. Furthermore, LAFF is efficient, requiring at most 317 ms per suggestion. [less ▲]

Demystifying Hidden Sensitive Operations in Android apps

in ACM Transactions on Software Engineering and Methodology (2022)

Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction and Clustering

in ACM Transactions on Software Engineering and Methodology (2022)

Deep neural networks (DNNs) have demonstrated superior performance over classical machine learning to support many features in safety-critical systems. Although DNNs are now widely used in such systems (e ... [more ▼]

Deep neural networks (DNNs) have demonstrated superior performance over classical machine learning to support many features in safety-critical systems. Although DNNs are now widely used in such systems (e.g., self driving cars), there is limited progress regarding automated support for functional safety analysis in DNN-based systems. For example, the identification of root causes of errors, to enable both risk analysis and DNN retraining, remains an open problem. In this paper, we propose SAFE, a black-box approach to automatically characterize the root causes of DNN errors. SAFE relies on a transfer learning model pre-trained on ImageNet to extract the features from error-inducing images. It then applies a density-based clustering algorithm to detect arbitrary shaped clusters of images modeling plausible causes of error. Last, clusters are used to effectively retrain and improve the DNN. The black-box nature of SAFE is motivated by our objective not to require changes or even access to the DNN internals to facilitate adoption. Experimental results show the superior ability of SAFE in identifying different root causes of DNN errors based on case studies in the automotive domain. It also yields significant improvements in DNN accuracy after retraining, while saving significant execution time and memory when compared to alternatives. [less ▲]

iBiR: Bug Report driven Fault Injection

in ACM Transactions on Software Engineering and Methodology (2022)

Predicting Patch Correctness Based on the Similarity of Failing Test Cases

in ACM Transactions on Software Engineering and Methodology (2022)

An Empirical Study on Data Distribution-Aware Test Selection for Deep Learning Enhancement

in ACM Transactions on Software Engineering and Methodology (2022)

Similar to traditional software that is constantly under evolution, deep neural networks (DNNs) need to evolve upon the rapid growth of test data for continuous enhancement, e.g., adapting to distribution ... [more ▼]

Similar to traditional software that is constantly under evolution, deep neural networks (DNNs) need to evolve upon the rapid growth of test data for continuous enhancement, e.g., adapting to distribution shift in a new environment for deployment. However, it is labor-intensive to manually label all the collected test data. Test selection solves this problem by strategically choosing a small set to label. Via retraining with the selected set, DNNs will achieve competitive accuracy. Unfortunately, existing selection metrics involve three main limitations: 1) using different retraining processes; 2) ignoring data distribution shifts; 3) being insufficiently evaluated. To fill this gap, we first conduct a systemically empirical study to reveal the impact of the retraining process and data distribution on model enhancement. Then based on our findings, we propose a novel distribution-aware test (DAT) selection metric. Experimental results reveal that retraining using both the training and selected data outperforms using only the selected data. None of the selection metrics perform the best under various data distributions. By contrast, DAT effectively alleviates the impact of distribution shifts and outperforms the compared metrics by up to 5 times and 30.09% accuracy improvement for model enhancement on simulated and in-the-wild distribution shift scenarios, respectively. [less ▲]

Taming Reflection: An Essential Step Toward Whole-program Analysis of Android Apps

in ACM Transactions on Software Engineering and Methodology (2021), 30(3), 1-36

Android developers heavily use reflection in their apps for legitimate reasons. However, reflection is also significantly used for hiding malicious actions. Unfortunately, current state-of-the-art static ... [more ▼]

Android developers heavily use reflection in their apps for legitimate reasons. However, reflection is also significantly used for hiding malicious actions. Unfortunately, current state-of-the-art static analysis tools for Android are challenged by the presence of reflective calls which they usually ignore. Thus, the results of their security analysis, e.g., for private data leaks, are incomplete, given the measures taken by malware writers to elude static detection. We propose a new instrumentation-based approach to address this issue in a non-invasive way. Specifically, we introduce to the community a prototype tool called DroidRA, which reduces the resolution of reflective calls to a composite constant propagation problem and then leverages the COAL solver to infer the values of reflection targets. After that, it automatically instruments the app to replace reflective calls with their corresponding Java calls in a traditional paradigm. Our approach augments an app so that it can be more effectively statically analyzable, including by such static analyzers that are not reflection-aware. We evaluate DroidRA on benchmark apps as well as on real-world apps, and demonstrate that it can indeed infer the target values of reflective calls and subsequently allow state-of-the-art tools to provide more sound and complete analysis results. [less ▲]

What You See is What it Means! Semantic Representation Learning of Code based on Visualization

in ACM Transactions on Software Engineering and Methodology (2021)

Recent successes in training word embeddings for NLP tasks have encouraged a wave of research on representation learning for sourcecode, which builds on similar NLP methods. The overall objective is then ... [more ▼]

Recent successes in training word embeddings for NLP tasks have encouraged a wave of research on representation learning for sourcecode, which builds on similar NLP methods. The overall objective is then to produce code embeddings that capture the maximumof program semantics. State-of-the-art approaches invariably rely on a syntactic representation (i.e., raw lexical tokens, abstractsyntax trees, or intermediate representation tokens) to generate embeddings, which are criticized in the literature as non-robustor non-generalizable. In this work, we investigate a novel embedding approach based on the intuition that source code has visualpatterns of semantics. We further use these patterns to address the outstanding challenge of identifying semantic code clones. Wepropose theWySiWiM(“What You See Is What It Means”) approach where visual representations of source code are fed into powerfulpre-trained image classification neural networks from the field of computer vision to benefit from the practical advantages of transferlearning. We evaluate the proposed embedding approach on the task of vulnerable code prediction in source code and on two variationsof the task of semantic code clone identification: code clone detection (a binary classification problem), and code classification (amulti-classification problem). We show with experiments on the BigCloneBench (Java), Open Judge (C) that although simple, ourWySiWiMapproach performs as effectively as state of the art approaches such as ASTNN or TBCNN. We also showed with datafrom NVD and SARD thatWySiWiMrepresentation can be used to learn a vulnerable code detector with reasonable performance(accuracy∼90%). We further explore the influence of different steps in our approach, such as the choice of visual representations or theclassification algorithm, to eventually discuss the promises and limitations of this research direction. [less ▲]

Killing Stubborn Mutants with Symbolic Execution

in ACM Transactions on Software Engineering and Methodology (2021), 30(2), 191--1923