Fig 3A illustrates the predictions on one replicate (combining all five of its folds, with each serving separately as test data) and Fig 3B summarizes the resulting area-under-ROC-curve (AUC) over all 200 replicates (computing AUC only on test data).

Nonetheless, even with a rigorous 200-replicate fivefold cross-validation, a mean AUC of 0.83 (standard deviation of 0.10) was observed, indicating that antibody features are highly and robustly predictive of high vs. low ADCP activity.

Despite the reduction in data considered, Fig 3C and 3D shows that the resulting performance with the filtered feature set is comparable to that with the complete feature set, with a mean AUC of 0.84 (standard deviation 0.10).

(AF) Prediction results by ZOO-replicate fivefold cross-validation, illustrating PLR values (>0.5 predicted high ADCP; <O.5 predicted low) for one replicate (A,C,E) and providing area under the ROC curve ( AUC ) over all 200 replicates (B,D,F).

(AF) Prediction results by ZOO-replicate fivefold cross-validation, illustrating PLR values (>0.5 predicted high ADCP; <O.5 predicted low) for one replicate (A,C,E) and providing area under the ROC curve ( AUC ) over all 200 replicates (B,D,F).

The method was bench marked on a dataset of 6,282 pairs of known interacting ligand-target pairs reaching a 0.96 of area under the Receiver Operating Characteristic curve (AUC) when using the drug names as an input feature for the classifier, and a 0.70 of AUC for “anonymous” compounds or compounds not present in the training set.

The AUC was excellent when using drug names and scores as input feature for the predictions.

When only the scores of the predictions were used (that is, treating the compound as anonymous), there was a clear decrease in the AUC suggesting that the method performs better for already known chemical entities rather than for new unseen compounds.

The RFC correctly recalled 66% of the pairs with a precision of 0.73 and an AUC of 0.71 using a 10-fold cross validation (Table 1).

However, by using the DrugBank ID as an input feature, the accuracy of nAnnoLyze dramatically improves to a 0.93 precision, 0.93 recall and a 0.97 of AUC (Fig.

Using ROC analysis (1ms timescale, 10-fold cross-validated), we find that spikes are predicted with an area-under-the-curve ( AUC ) of 0.71:0.01 for Model 1, while Model 2 yields 0.72:0.01, 0.74:0.01, and 0.77:0.01 as the input amplitude increases from 0.5 a, to 1.00 a, to 1.5 a.

However, Model 2 provides better spike prediction ( AUC ) in all cases (paired t-tests, p = 0.01, p<10_5, p<10_3).

With few inputs, the post-synaptic firing is dominated by post-spike effects and timing of postsynaptic spikes is predicted relatively inaccurately ( AUC = 0.78 for this example with 4 inputs).

With an increasing fraction of inputs included in the model, their contribution to spike prediction increases, and timing of individual spikes is predicted with progressively increasing accuracy ( AUC = 0.82, 0.86, and 0.98 for 256, 512, and 1024 inputs, respectively).

B) Model accuracy: Receiver operating characteristic (ROC) curves for the example cell and area under the curve ( AUC ) for all cells from A. Curves show the cross-validated false positive rate (FPR) vs true positive rate (TPR) for spike detection in 1ms bins.

Both networks show high AUC values and even better accuracy (see Table 1).

In (A) the area under the ROC curve ( AUC ) was calculated based on edge frequencies of the samples.

The second and third column show performance of the networks in terms of accuracy (ACC) and area under curve ( AUC ).