Skip to content

Commit 46f5707

Browse files
dvojtisedvojtise
authored
Bump Studio to 3.4.0 (#236)
* bump GEMOC Studio to 3.4.0.qualifier * add script to monitor process memory in the docker environment * put jfx in better place (docker image) * Update vendor name contributes to #235 Signed-off-by: Didier Vojtisek <[email protected]>
1 parent 92600ca commit 46f5707

File tree

55 files changed

+220
-76
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

55 files changed

+220
-76
lines changed

‎dev_support/full_compilation/pom.xml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
<groupId>org.eclipse.gemoc</groupId>
77
<artifactId>complete-studio</artifactId>
88
<packaging>pom</packaging>
9-
<version>3.3.0-SNAPSHOT</version>
9+
<version>3.4.0-SNAPSHOT</version>
1010
<name>Complete GEMOC Studio compilation</name>
1111

1212
<properties>

‎dev_support/jenkins/docker/Dockerfile‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ RUN yum install -y yum-utils && \
3232
RUN rpm -v --import http://li.nux.ro/download/nux/RPM-GPG-KEY-nux.ro
3333
RUN rpm -Uvh http://li.nux.ro/download/nux/dextop/el7/x86_64/nux-dextop-release-0-5.el7.nux.noarch.rpm
3434

35+
3536
RUN yum update -y \
3637
&& yum install -y \
3738
autoconf \
@@ -66,7 +67,7 @@ RUN yum update -y \
6667
perl \
6768
perl-LDAP \
6869
python-gtk-vnc \
69-
python36u\
70+
python36u python36u-libs python36u-devel python36u-pip\
7071
rpm-build \
7172
strace \
7273
subversion \
@@ -117,10 +118,10 @@ ENV M2_HOME /usr/share/maven
117118
ENV HOME=/home/jenkins
118119

119120
RUN wget https://gluonhq.com/download/javafx-11-0-2-sdk-linux -O ${HOME}/javafx-11-0-2-sdk-linux.zip
120-
RUN unzip ${HOME}/javafx-11-0-2-sdk-linux.zip -d ${HOME}
121+
RUN unzip ${HOME}/javafx-11-0-2-sdk-linux.zip -d /usr/share/
121122
RUN rm ${HOME}/javafx-11-0-2-sdk-linux.zip
122123

123-
ENV JAVAFX_HOME=${HOME}/javafx-sdk-11.0.2
124+
ENV JAVAFX_HOME=/usr/share/javafx-sdk-11.0.2
124125

125126
RUN ln -s /usr/bin/git /usr/local/bin/git \
126127
&& ln -s /bin/bash /usr/local/bin/hipp_shell \
@@ -136,6 +137,11 @@ RUN mkdir -p ${HOME}/.vnc && chmod -R 775 ${HOME} \
136137
COPY scripts/xstartup_mutter.sh ${HOME}/.vnc/xstartup.sh
137138
RUN chmod 755 ${HOME}/.vnc/xstartup.sh
138139

140+
# install memory monitor script file
141+
COPY scripts/memory-monitor/memory-monitor-per-process.py ${HOME}/memory-monitor-per-process.py
142+
RUN python3 -m pip install psutil
143+
RUN chmod 755 ${HOME}/memory-monitor-per-process.py
144+
139145
# explicitly set locale
140146
ENV LANG=en_US.UTF-8
141147

‎dev_support/jenkins/docker/scripts/memory-monitor/README.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
how to start the service:
2+
3+
4+
### Install Miniconda
5+
6+
**This step is only necessary if you don't have conda installed**:
7+
- download the Miniconda installer for your OS [here](https://docs.conda.io/en/latest/miniconda.html)
8+
- run the installer following the instructions
9+
[here](https://conda.io/projects/conda/en/latest/user-guide/install/index.html#regular-installation)
10+
depending on your OS.
11+
12+
### Create conda environment
13+
14+
```sh
15+
# Create a conda environment with the required packages for this notebook:
16+
conda env create -f environment.yml
17+
# Activate your conda environment
18+
conda activate collect-monitor-service
19+
```

‎dev_support/jenkins/docker/scripts/memory-monitor/environment.yml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
name: memory-monitor
2+
channels:
3+
- conda-forge
4+
dependencies:
5+
- python=3.7
6+
- pip
7+
- psutil

‎dev_support/jenkins/docker/scripts/memory-monitor/memory-monitor-per-process.py‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
#! /usr/bin/python3
2+
3+
# script derived and adapted from https://github.com/giampaolo/psutil/blob/master/scripts/procsmem.py
4+
5+
from __future__ import print_function
6+
import sys
7+
import time
8+
import psutil
9+
import copy
10+
11+
12+
13+
if not (psutil.LINUX or psutil.MACOS or psutil.WINDOWS):
14+
sys.exit("platform not supported")
15+
16+
17+
def convert_bytes(n):
18+
symbols = ('K', 'M', 'G', 'T', 'P', 'E', 'Z', 'Y')
19+
prefix = {}
20+
for i, s in enumerate(symbols):
21+
prefix[s] = 1 << (i + 1) * 10
22+
for s in reversed(symbols):
23+
if n >= prefix[s]:
24+
value = float(n) / prefix[s]
25+
return '%.1f%s' % (value, s)
26+
return "%sB" % n
27+
28+
def printProc(p):
29+
templ = "%-7s %-7s %7s %7s %7s %7s %7s"
30+
cmd = " ".join(p._info["cmdline"])[:256] if p._info["cmdline"] else ""
31+
line = templ % (
32+
p.pid,
33+
p._info["username"][:7] if p._info["username"] else "",
34+
convert_bytes(p._uss),
35+
convert_bytes(p._pss) if p._pss != "" else "",
36+
convert_bytes(p._swap) if p._swap != "" else "",
37+
convert_bytes(p._rss),
38+
cmd,
39+
)
40+
print(line)
41+
42+
def printAllMax(procs):
43+
procs.sort(key=lambda p: p['uss'])
44+
print("=" * 78)
45+
templ = "%-7s %-7s %7s %7s %7s %7s %7s"
46+
t = time.localtime()
47+
current_time = time.strftime("%H:%M:%S", t)
48+
print("Memory information used by processes at when last seen at maximum unique set size (USS) ", current_time)
49+
print(templ % ("PID", "User", "max(USS)", "PSS", "Swap", "RSS", "Cmdline"))
50+
print("=" * 78)
51+
# for p in procs[:86]:
52+
for p in procs:
53+
printProc(p['proc'])
54+
55+
def main():
56+
ad_pids = []
57+
procs = []
58+
59+
prevProcsMap = {}
60+
procsMap = {}
61+
62+
63+
# i = 1
64+
# while i < 1000:
65+
# print("loop ", i)
66+
# i += 1
67+
while True:
68+
newMaxObserved = False
69+
# procsMap = {p.pid: p.info for p in psutil.process_iter()}
70+
for p in psutil.process_iter():
71+
with p.oneshot():
72+
try:
73+
mem = p.memory_full_info()
74+
info = p.as_dict(["cmdline", "username"])
75+
except psutil.AccessDenied:
76+
ad_pids.append(p.pid)
77+
except psutil.NoSuchProcess:
78+
pass
79+
else:
80+
p._uss = mem.uss
81+
p._rss = mem.rss
82+
if not p._uss:
83+
continue
84+
p._pss = getattr(mem, "pss", "")
85+
p._swap = getattr(mem, "swap", "")
86+
p._info = info
87+
if int(p._pid) in prevProcsMap:
88+
if prevProcsMap[int(p._pid)]['uss'] < p._uss:
89+
print('New max USS for pid ', p._pid, ' : USS=', prevProcsMap[int(p._pid)]['uss'], '->', p._uss)
90+
# printProc(p)
91+
procsMap[int(p._pid)] = { "uss": p._uss, "proc": p}
92+
prevProcsMap[int(p._pid)] = { "uss": p._uss, "proc": p}
93+
newMaxObserved = True
94+
else:
95+
newMaxObserved = True
96+
print('New pid : ', p._pid, ': USS=', p._uss)
97+
# printProc(p)
98+
procsMap[int(p._pid)] = { "uss": p._uss, "proc": p}
99+
prevProcsMap[int(p._pid)] = { "uss": p._uss, "proc": p}
100+
# procs.append(p)
101+
#prevProcsMap = copy.deepcopy(procsMap)
102+
if newMaxObserved:
103+
printAllMax(list(prevProcsMap.values()))
104+
# keep only active pid
105+
for pid in [pid for pid in prevProcsMap.keys() if pid not in procsMap.keys()]:
106+
prevProcsMap.pop(pid)
107+
procsMap.clear()
108+
time.sleep(1)
109+
110+
111+
if __name__ == '__main__':
112+
sys.exit(main())

‎docs/org.eclipse.gemoc.studio.doc/pom.xml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55

66
<groupId>org.eclipse.gemoc.documentation</groupId>
77
<artifactId>gemoc-studio-documentation</artifactId>
8-
<version>3.3.0-SNAPSHOT</version>
8+
<version>3.4.0-SNAPSHOT</version>
99
<!-- <packaging>jdocbook</packaging>-->
1010
<packaging>jar</packaging>
1111

‎gemoc_studio/plugins/gemoc_studio-eclipse-bom/pom.xml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
<modelVersion>4.0.0</modelVersion>
66
<groupId>org.eclipse.gemoc.gemoc-studio</groupId>
77
<artifactId>gemoc_studio-eclipse-bom</artifactId>
8-
<version>3.3.0-SNAPSHOT</version>
8+
<version>3.4.0-SNAPSHOT</version>
99
<packaging>pom</packaging>
1010
<name>GEMOC Studio Eclipse Product BOM</name>
1111
<description>GEMOC Studio Eclipse Product Bill of Materials</description>

‎gemoc_studio/plugins/org.eclipse.gemoc.gemoc_studio.branding/META-INF/MANIFEST.MF‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@ Manifest-Version: 1.0
22
Bundle-ManifestVersion: 2
33
Bundle-Name: GEMOC Studio
44
Bundle-SymbolicName: org.eclipse.gemoc.gemoc_studio.branding;singleton:=true
5-
Bundle-Version: 3.3.0.qualifier
6-
Bundle-Vendor: GEMOC
5+
Bundle-Version: 3.4.0.qualifier
6+
Bundle-Vendor: Eclipse GEMOC Project
77
Bundle-Localization: plugin
88
Require-Bundle: org.eclipse.core.runtime,
99
org.eclipse.ui,

‎gemoc_studio/plugins/org.eclipse.gemoc.gemoc_studio.branding/pom.xml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
<parent>
99
<groupId>org.eclipse.gemoc.gemoc_studio</groupId>
1010
<artifactId>org.eclipse.gemoc.gemoc_studio.parent</artifactId>
11-
<version>3.3.0-SNAPSHOT</version>
11+
<version>3.4.0-SNAPSHOT</version>
1212
<relativePath>../..</relativePath>
1313
</parent>
1414

‎gemoc_studio/plugins/org.eclipse.gemoc.gemoc_studio.headless/META-INF/MANIFEST.MF‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@ Manifest-Version: 1.0
22
Bundle-ManifestVersion: 2
33
Bundle-Name: GEMOC Studio Headless
44
Bundle-SymbolicName: org.eclipse.gemoc.gemoc_studio.headless;singleton:=true
5-
Bundle-Version: 3.3.0.qualifier
6-
Bundle-Vendor: Eclipse
5+
Bundle-Version: 3.4.0.qualifier
6+
Bundle-Vendor: Eclipse GEMOC Project
77
Automatic-Module-Name: org.eclipse.gemoc.headless
88
Bundle-RequiredExecutionEnvironment: JavaSE-1.8
99
Require-Bundle: org.eclipse.equinox.app,

0 commit comments

Comments
 (0)